Skip to yearly menu bar Skip to main content


Poster

MANUS: Markerless Grasp Capture using Articulated 3D Gaussians

Chandradeep Pokhariya · Ishaan Shah · Angela Xing · Zekun Li · Kefan Chen · Avinash Sharma · Srinath Sridhar


Abstract:

Understanding how we grasp objects with our hands has important applications in areas like robotics and mixed reality.However, this challenging problem requires accurate modeling of the contact between hands and objects.To capture grasps, existing methods use skeletons, meshes, or parametric models that can cause misalignments resulting in inaccurate contacts. We present MANUS, a method for MArkerless Hand-Object Grasp Capture using Articulated 3D GaussiansWe build a novel articulated 3D Gaussians representation that extends 3D Gaussian splatting for high-fidelity representation of articulating hands. Since our representation uses Gaussian primitives, it enables us to efficiently and accurately estimate contacts between the hand and the object. For the most accurate results, our method requires tens of camera views that current datasets do not provide.We therefore build MANUS-Grasps, a new dataset that contains hand-object grasps viewed from 53 cameras across 30+ scenes, 3 subjects, and comprising over 7M frames. In addition to extensive qualitative results, we also show that our method outperforms others on a quantitative contact evaluation method that uses paint transfer from the object to the hand.

Live content is unavailable. Log in and register to view live content