Toward Real-world BEV Perception: Depth Uncertainty Estimation via Gaussian Splatting

Bird's-eye view (BEV) perception has gained significant attention because it provides a unified representation to fuse multiple view images and enables a wide range of downstream autonomous driving tasks, such as forecasting and planning. However, the task is challenging because it is inherently an ill-posed problem due to the lack of depth information. Moreover, fusing multi-view images into a unified representation without depth cues becomes more challenging.Recent grid-based methods formulate BEV perception as query learning to bypass explicit depth estimation. While we observe promising advancements in this paradigm, they still fall short of real-world applications because they lack uncertainty modeling and are computationally expensive.In this work, we revisit depth-based methods and endow them with uncertainty awareness. Specifically, we calculate the variance of the depth distribution to represent how objects are spatially dispersed around their mean depth. In addition, the proposed model learns a soft depth mean and implicitly captures the spatial extent of objects. We transform the depth distribution into 3D Gaussians and utilize the rasterization technique to form uncertainty-aware BEV features.We evaluate our method on the nuScenes dataset, achieving state-of-the-art performance compared to depth-based methods. Notably, our model provides significant advantages in speed—running 2x faster—and in memory efficiency, using 0.3x less memory compared to grid-sampling-based methods, while achieving competitive performance with only a 0.7% IoU difference.