Volumetric Video in the Real World

Volumetric video, which encodes a time-varying 3D scene into a unified representation for novel-view synthesis of dynamic contents, has long been a grand challenge for achieving immersive experiences. High-quality volumetric video enables new and immersive applications, such as 3D video conferencing, 3D telepresence, and virtual tutoring in XR. Recent volumetric representation enables fast and high-quality reconstruction of dynamic 3D scenes. As such, our tutorial summarizes practical challenges towards generating and distributing volumetric video in the real world. Specifically, invited talks in this tutorial will cover: (1) compression and performance optimization for 4D reconstruction, such as dynamic Gaussian splatting, quantization, and autoencoders; (2) volumetric video reconstruction from single or sparse-view captures; (3) reconstruction of indoor and urban scenes with dynamic content; (4) reconstruction and playback of dynamic 4D humans in the real-world; and (5) integration of volumetric video with vision-language models for other applications. Challenges across video domains, such as dynamic humans, automotive video, and synthetically generated video, will be thoroughly discussed.