Spotting the Unexpected (STU): A 3D LiDAR Dataset for Anomaly Segmentation in Autonomous Driving

For safe operation, autonomous vehicles (AVs) must detect and handle unexpected objects or anomalies on the road. While anomaly detection and segmentation have been explored in 2D images, a gap remains for similar tasks in 3D LiDAR point clouds. Existing datasets lack high-quality multimodal data typically found in AVs. This paper presents a novel dataset for anomaly segmentation in driving scenarios. To the best of our knowledge, it is the first publicly available dataset focused on road anomaly segmentation with dense 3D semantic labeling, incorporating both LiDAR and camera data, as well as sequential information to enable anomaly detection across various ranges. This capability is critical for the safe navigation of autonomous vehicles. We adapted and benchmarked several baseline models for 3D segmentation, highlighting the challenges of 3D anomaly detection in driving environments. Our dataset and evaluation code will be openly accessible, facilitating testing and performance comparison across diverse approaches.