Skip to yearly menu bar Skip to main content


Open-World Semantic Segmentation Including Class Similarity

Matteo Sodano · Federico Magistri · Lucas Nunes · Jens Behley · Cyrill Stachniss

Arch 4A-E Poster #294
[ ]
Wed 19 Jun 10:30 a.m. PDT — noon PDT


Interpreting camera data is key for autonomously-acting systems, such as autonomous vehicles. Vision systems that operate in real-world environments must be able to understand their surroundings and need the ability to deal with novel situations. This paper tackles open-world semantic segmentation, i.e., the variant of interpreting camera data in which objects occur that have not been seen during training. We propose a novel approach that performs accurate closed-world semantic segmentation and, at the same time, can identify new categories without requiring any additional training data. Our approach additionally provides a similarity measure for every newly discovered class in an image to a known category, which is useful information in further downstream tasks such as planning or mapping. Through extensive experiments, we show that our model achieves state-of-the-art results on classes known from training data as well as for anomaly segmentation and can distinguish between different unknown classes.

Live content is unavailable. Log in and register to view live content