UPPET: Unified Pedestrian Pose Estimation in Thermal Imaging

Human or pedestrian pose estimation is a crucial task in many real-world applications, such as human-computer interaction, sports analytics, or surveillance. Currently, research focuses primarily on human pose estimation on imagery captured in the visible spectrum. However, thermal cameras gain increasingly importance, as these sensors are able to observe persons even during complete darkness or during difficult weather conditions. In this work, we propose two novel datasets for human pose estimation, namely CAMEL-P and TPE, in thermal data. In total, we contribute annotations for 2,926 images resulting in 25,951 poses for CAMEL-P and 14,321 images, with a total of 52,563 poses for TPE. Furthermore, we extend the annotation of 14,286 existing poses for the OpenThermalPose, and together with LLVIP-Pose, we create the UPPET dataset composed of four datasets to enable cross-dataset experiments. This enables for the first time to assess the generalization ability of state-of-the-art models across various thermal sensors with diverse image resolutions and across different scenarios. Our experiments reveal strong results when training and evaluating state-of-the-art human pose estimation methods on the same data sources. However, our study indicates a significant drop in accuracy for generalization scenarios, highlighting the need for further research to enhance the robustness of human pose methods in thermal imaging. The datasets and baseline code will be released upon publication on: https://github.com/MickaelCormier/uppet