Skip to yearly menu bar Skip to main content


ChatScene: Knowledge-Enabled Safety-Critical Scenario Generation for Autonomous Vehicles

Jiawei Zhang · Chejian Xu · Bo Li

Arch 4A-E Poster #82
[ ]
Thu 20 Jun 5 p.m. PDT — 6:30 p.m. PDT


We present ChatScene, a unified and effective framework that leverages the capabilities of Large Language Models (LLMs) to guide the generation of safety-critical scenarios for autonomous vehicles. This framework distinctively transforms textually described scenarios generated by LLMs into domain-specific languages, which then generate actual code for prediction and control in simulators, facilitating the creation of diverse and complex scenarios within the CARLA simulation environment. A key part of our approach is a comprehensive knowledge retrieval component, which efficiently translates specific textual descriptions into corresponding domain-specific code snippets by training a knowledge database containing the scenario description and code pairs. Extensive experimental results underscore the efficacy of ChatScene in improving the safety of autonomous vehicles. For instance, the scenarios generated by ChatScene show a 15% increase in collision rates compared to state-of-the-art baselines when tested against different reinforcement learning-based ego vehicles. Furthermore, we show that by using our generated safety-critical scenarios to fine-tune different autonomous agents, they can achieve a 9% reduction in collision rates, surpassing current SOTA methods. ChatScene effectively bridges the gap between textual descriptions of traffic scenarios and practical CARLA simulations.

Live content is unavailable. Log in and register to view live content