Insightful Instance Features for 3D Instance Segmentation

Recent 3D Instance Segmentation methods typically encode hundreds of instance-wise candidates with instance-specific information in various ways and refine them into final masks.However, they have yet to fully explore the benefit of these candidates.They overlook the valuable cues encoded in multiple candidates that represent different parts of the same instance, resulting in fragments.Also, they often fail to capture the precise spatial range of 3D instances, primarily due to inherent noises from sparse and unordered point clouds.In this work, to address these challenges, we propose a novel instance-wise knowledge enhancement approach.We first introduce an Instance-wise Knowledge Aggregation to associate scattered single instance details by optimizing correlations among candidates representing the same instance.Moreover, we present an Instance-wise Structural Guidance to enhance the spatial understanding of candidates using structural cues from ambiguity-reduced features.Here, we utilize a simple yet effective truncated singular value decomposition algorithm to minimize inherent noises of 3D features.In our extensive experiments on large-scale benchmarks, ScanNetV2, ScanNet200, S3DIS, and STPLS3D, our method outperforms existing works.We also demonstrate the effectiveness of our modules based on both kernel and transformer architectures.