Den-TP: A Density-Balanced Data Curation and Evaluation Framework for Trajectory Prediction

Trajectory prediction in autonomous driving has traditionally been studied from a model-centric perspective. However, existing datasets exhibit a strong long-tail distribution in scenario density, the number of agents per scenario, where common low-density cases dominate and safety-critical high-density cases are severely underrepresented. This imbalance limits model robustness and hides failure modes when standard evaluations average errors across all scenarios. We revisit trajectory prediction from a data-centric angle and present Den-TP, a framework for density-aware dataset curation and evaluation. Den-TP first partitions data into density-conditioned regions using agent count as a lightweight, dataset-agnostic proxy for interaction complexity. It then applies gradient-based utilities with a submodular selection objective to choose representative samples within each region while explicitly rebalancing across densities. The resulting subset reduces dataset size by 50\% yet preserves overall performance and significantly improves robustness in high-density scenarios. We further introduce density-conditioned evaluation protocols that reveal long-tail failure modes overlooked by conventional metrics. Experiments on Argoverse 1 and 2 with state-of-the-art models show that robust trajectory prediction hinges not only on data scale, but also on balancing scenario density.