Universal Domain Adaptation for Semantic Segmentation

Unsupervised domain adaptation for semantic segmentation (UDA-SS) aims to transfer knowledge from labeled synthetic data (source) to unlabeled real-world data (target). Traditional UDA-SS methods work on the assumption that the category settings between the source and target domains are known in advance. However, in real-world scenarios, the category settings of the source and target are unknown due to the lack of labels in the target, resulting in the existence of target-private or source-private classes. Traditional UDA-SS methods struggle with this change, leading to negative transfer and performance degradation. To address these issues, we propose Universal Domain Adaptation for Semantic Segmentation (UniDA-SS) for the first time to achieve good performance even when the category settings of source and target are unknown. We defined the problem in the UniDA-SS scenario as that the confidence score of common classes in the target is lowered, which leads to confusion with private classes. To solve this problem, we propose two methods.The first method is Domain-Specific Prototype-based Distinction, which divides a class into two prototypes, each with domain-specific features, and then weights the common class based on the fact that if it is common, it will be similar to both prototypes. The second method is Target-based Image Matching (TIM), which finds the source image with the most common class based on the target pseudo label and trains in a batch to sample as many common class pixels as possible from the target side to increase the confidence score of the common class. We propose a new benchmark for the UniDA-SS scenario and show that it effectively solves the UniDA-SS scenario problem compared to the baseline through various experiments.