Skip to yearly menu bar Skip to main content


KP-RED: Exploiting Semantic Keypoints for Joint 3D Shape Retrieval and Deformation

Ruida Zhang · Chenyangguang Zhang · Yan Di · Fabian Manhardt · Xingyu Liu · Federico Tombari · Xiangyang Ji

Arch 4A-E Poster #96
[ ]
Fri 21 Jun 10:30 a.m. PDT — noon PDT


In this paper, we present KP-RED, a unified KeyPoint-driven REtrieval and Deformation framework that takes object scans as input and jointly retrieves and deforms the most geometrically similar CAD models from a pre-processed database to tightly match the target.Unlike existing dense matching based methods that typically struggle with noisy partial scans, we propose to leverage category-consistent sparse keypoints to naturally handle both full and partial object scans. Specifically, we first employ a lightweight retrieval module to establish a keypoint-based embedding space, measuring the similarity among objects by dynamically aggregating deformation-aware local-global features around extracted keypoints.Objects that are close in the embedding space are considered similar in geometry.Then we introduce the neural cage-based deformation module that estimates the influence vector of each keypoint upon cage vertices inside its local support region to control the deformation of the retrieved shape.Extensive experiments on the synthetic dataset PartNet and the real-world dataset Scan2CAD demonstratethat KP-RED surpasses existing state-of-the-art approaches by a large margin.Codes and trained models will be released soon.

Live content is unavailable. Log in and register to view live content