Generative 3D Part Assembly via Part-Whole-Hierarchy Message Passing

Generative 3D part assembly involves understanding partrelationships and predicting their 6-DoF poses for assembling a realistic 3D shape. Prior work often focus on thegeometry of individual parts, neglecting part-whole hierarchies of objects. Leveraging two key observations: 1)super-part poses provide strong hints about part poses, and2) predicting super-part poses is easier due to fewer superparts, we propose a part-whole-hierarchy message passingnetwork for efficient 3D part assembly. We first introducesuper-parts by grouping geometrically similar parts withoutany semantic labels. Then we employ a part-whole hierarchical encoder, wherein a super-part encoder predicts latentsuper-part poses based on input parts. Subsequently, wetransform the point cloud using the latent poses, feeding itto the part encoder for aggregating super-part informationand reasoning about part relationships to predict all partposes. In training, only ground-truth part poses are required.During inference, the predicted latent poses of super-partsenhance interpretability. Experimental results on the PartNetdataset show that our method achieves state-of-the-art performance in part and connectivity accuracy and enables aninterpretable hierarchical part assembly. Code is availableat https://github.com/pkudba/3DHPA.