Open-World Objectness Modeling Unifies Novel Object Detection

The challenge in open-world object detection, as in many few- and zero-shot learning problems, is to generalize beyond the class distribution of the training data. We thus propose a general class-agnostic objectness measure to reduce bias toward labeled samples. To prevent previously unseen objects from being filtered as background or misclassified as known categories by classifers, we explicitly model the joint distribution of objectness and category labels using variational approximation. Without sufficient labeled data, minimizing the KL divergence between the estimated posterior and a static normal prior fails to converge, however. Theoretical analysis illuminates the root cause and motivates adopting a Gaussian prior with variance dynamically adapted to the estimated posterior as a surrogate. To further reduce misclassification, we introduce an energy-based margin loss that encourages unknown objects to move toward high-density regions of the distribution, thus reducing the uncertainty of unknown detections. We introduce an energy-based Open-World OBJectness modeling (OWOBJ) to boost novel object detection, especially in low-data settings. As a flexible plugin, OWOBJ outperforms baselines in Open-World, Few-Shot, and zero-shot Open-Vocabulary Object Detection. Code will be released upon acceptance.