提出首个交通异常理解视觉语言模型，支持细粒度分析与高效部署。
TAU-R1: Visual Language Model for Traffic Anomaly Understanding
- 分两层设计：轻量分类器粗筛异常，大模型生成详细事件解释。
- 在342段真实环岛视频上达优秀表现，覆盖2000+问答对。
- 适合智能交通系统研发者及交通行为分析研究者使用。
交通异常理解（TAU）对智能交通系统的交通安全至关重要。尽管近期视觉语言模型（VLMs）在视频理解方面表现出色，但因缺乏基准数据集和专用方法，TAU进展受限。为此，我们与印第安纳州卡姆尔市合作，构建了Roundabout-TAU数据集，包含342段真实环岛视频，标注超过2000个涵盖多维度交通异常理解的问答对。基于此，我们提出TAU-R1，一种双层视觉语言框架：第一层为轻量级异常分类器，实现粗粒度异常分类；第二层为大型异常推理器，生成详细事件摘要。为提升任务特定推理能力，引入两阶段训练策略：先进行分解问答增强的监督微调，再通过基于GRPO的后训练方法TAU-GRPO，结合专用于TAU的奖励函数优化。实验表明，TAU-R1在异常分类与推理任务上均表现优异，同时保持部署效率。数据集与代码已开源：https://github.com/siri-rouser/TAU-R1。
原文摘要 · Abstract (English)
Traffic Anomaly Understanding (TAU) is important for traffic safety in Intelligent Transportation Systems. Recent vision-language models (VLMs) have shown strong capabilities in video understanding. However, progress on TAU remains limited due to the lack of benchmarks and task-specific methodologies. To address this limitation, we introduce Roundabout-TAU, a dataset constructed from real-world roundabout videos collected in collaboration with the City of Carmel, Indiana. The dataset contains 342 clips and is annotated with more than 2,000 question-answer pairs covering multiple aspects of traffic anomaly understanding. Building on this benchmark, we propose TAU-R1, a two-layer vision-language framework for TAU. The first layer is a lightweight anomaly classifier that performs coarse anomaly categorisation, while the second layer is a larger anomaly reasoner that generates detailed event summaries. To improve task-specific reasoning, we introduce a two-stage training strategy consisting of decomposed-QA-enhanced supervised fine-tuning followed by TAU-GRPO, a GRPO-based post-training method with TAU-specific reward functions. Experimental results show that TAU-R1 achieves strong performance on both anomaly classification and reasoning tasks while maintaining deployment efficiency. The dataset and code are available at: https://github.com/siri-rouser/TAU-R1
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。