Beyond Reassembly: Fractured Object Recovery with Missing Parts

We propose a novel learning-based task named fractured object recovery. Unlike previous fractured object reassembly task that only targets aligning existing parts with overlaps, our task aims to not only reassemble irrelevant parts but also predict missing parts, resulting in a complete shape recovery immediately. Our task coincides with practical experiences, where the prior knowledge of similar shapes can be leverage in the reassembly process, such that even non-overlapping parts can be reasoned into adequate locations. We also present the first learning model for the proposed task by correlating features of both existing and missing parts using a transformer, where the latter is naturally represented as missing tokens. Hence, our model can jointly estimate the poses of the existing parts and predict the shapes of the missing parts. To facilitate the task, we introduce a new dataset based on the existing fractured object benchmark by imposing different configurations of missing parts. We perform extensive evaluations to demonstrate the performance of the proposed model over baselines. The results show that joint part reassembly and prediction can be made possible and also have mutual benefits, which we believe can inspire future research and favor real applications.