Extending Computer Vision to Hidden Objects: A Tutorial on Millimeter-Wave Imaging and Reconstruction of Occluded Scenes

Millimeter-wave (mmWave) signals, such as those used in 5G, 6G, and next-generation WiFi,
are a unique modality that can travel through many everyday occlusions (e.g. cardboard, fabric, fog, etc), allowing them to sense objects or scenes that are hidden from view. This unique capability has sparked recent interest in the computer vision community and beyond for using these signals to enable novel perception tasks with applications spanning autonomous driving, robotics, shipping and logistics, and more. The goal of this tutorial is to introduce audience members to this modality, and equip them with the knowledge needed to begin research in this area. We will cover both fundamental millimeter-wave imaging concepts, as well as recent, state-of-the-art methods. We will discuss different applications of millimeter-wave sensing, including a deep-dive into two areas: through-occlusion 3D object reconstruction, and all-weather scene understanding. We will additionally cover existing datasets, benchmarks, and tools so that audience members new to this area can begin research in this field. This tutorial is designed to be accessible for an audience with no prior millimeter-wave experience. Topics to be covered include:How millimeter wave signals differ from visible lightHow millimeter wave signals differ from other through-occlusion modalities (e.g., X-Ray, Ultrasound, etc)Various applications of mmWave sensing, including how they have been used in CV communityClassical methods for using mmWave signals to produce a 2D or 3D imageLimitations of classical mmWave imagingState-of-the-art methods for using mmWave signals to perform surface normal estimation for 3D object reconstructionState-of-the-art methods for using mmWave signals for complete scene reconstruction, segmentation, and object detectionHow researchers can get started in this area, including existing datasets, benchmarks, and tools