Improved Out-of-Distribution Detection with Additive Angular Margin Loss

Deep Neural Networks (DNNs) are prone to overconfident predictions when encountering out-of-distribution (OoD) examples—data points that do not belong to the true underlying distribution of the training set. Therefore identifying these OoD inputs is critical for safe deployment of DNNs.OoD detection relies on the fact that OoD examples map closer to the decision boundary than in-distribution data. We hypothesize that enforcing larger margins will constrain DNNs to map in-distribution data further from the decision boundary. Thus, enforcing margins may improve the separability of in-distribution from OoD data.In this paper, we propose using Additive Angular Margin (AAM) to enforce DNNs to learn large margins. However, we find that DNNs trained with AAM have very poor calibration leading to poor OoD detection performance. This is because AAM projects the latent space representation onto a hyper-sphere resulting in a loss of scaling information. Thus, we propose using a scaled version of AAM during inference. The proposed modification to AAM skips the projection step during inference.The modified Additive Angular Margin approach achieves 3.42% and 6.26% improvement in AUROC and AUPR metrics respectively, on various existing OoD detection schemes such as ODIN, Mahalanobis and Energy Based OoD detectors. The experimental results suggest that AAM is a generic tool that can be used to improve OoD detection performance.