PSA-SSL: Pose and Size-aware Self-Supervised Learning on LiDAR Point Clouds

Self-supervised learning (SSL) on 3D point clouds has the potential to learn feature representations that can transfer to diverse sensors and multiple downstream perception tasks. However, recent SSL approaches fail to define pretext tasks that retain geometric information such as object pose and scale, which can be detrimental to the performance of downstream localization and geometry-sensitive 3D scene understanding tasks, such as 3D semantic segmentation and 3D object detection. Further, no methods exist that exhibit transferable performance across different LiDAR beam patterns. We propose PSA-SSL, a novel extension to point cloud SSL that learns LiDAR pattern-agnostic and object pose and size-aware (PSA) features. Our approach defines 1) a self-supervised bounding box regression pretext task, which retains object pose and size information, and 2) a LiDAR beam pattern augmentation on input point clouds, which encourages learning sensor-agnostic features. Our experiments demonstrate that with a single pre-trained model, our light-weight yet effective extensions achieve significant improvements on 3D semantic segmentation with limited labels (up to \textbf{+2.66 mIoU} for DepthContrast and \textbf{+2.39 mIoU} for SegContrast) across popular autonomous driving datasets (Waymo, NuScenes, SemanticKitti). Moreover, our approach outperforms other state-of-the-art SSL methods on 3D semantic segmentation and shows improvements on 3D object detection when transferring to different LiDAR sensors. Our code will be released on (anonymized link).