arXiv:2512.18878cs.CVcs.AI2025-12被引 1

构建多任务交通碰撞视频分析模型,实现精准识别与描述。

CrashChat: A Multimodal Large Language Model for Multitask Traffic Crash Video Analysis

  • 基于视频-语言大模型,采用任务解耦分组策略提升多任务学习效果。
  • 在碰撞识别上接近完美准确率,定位性能提升176%,预碰撞定位提升40%。
  • 适合自动驾驶安全研究、事故责任分析等实际场景使用。

自动化车祸视频分析对利用日益增长的驾驶视频数据开展交通安全研究和自动驾驶责任认定至关重要。由于车祸事件在视频中具有复杂的时空动态及多样化的分析需求,该任务涉及碰撞识别、时间定位和高层视频理解,极具挑战性。现有模型无法在统一框架内完成所有任务,且针对此类模型的有效训练策略仍不充分。为此,本文提出CrashChat,一个基于VideoLLaMA3的多模态大语言模型(MLLM),用于多任务交通碰撞分析。CrashChat通过指令微调获取领域知识,并采用新颖的多任务学习策略——任务解耦与分组,最大化组内及组间联合学习效益,同时缓解负迁移问题。在整合公开数据集上的数值实验表明,CrashChat在不同模型规模下均优于现有MLLMs和传统视觉方法,达到领先水平:碰撞识别接近完美准确率,碰撞定位性能提升176%,预碰撞定位提升40%。相比通用MLLMs,其在碰撞描述与推理任务中的文本准确率和内容覆盖率显著提升,BLEU分数提高0.18–0.41,ROUGE分数提高0.18–0.42。此外,CrashChat是可直接部署的端到端分析工具。相关数据集与代码已开源:https://github.com/Liangkd/CrashChat。

原文摘要 · Abstract (English)

Automating crash video analysis is essential to leverage the growing availability of driving video data for traffic safety research and accountability attribution in autonomous driving. Crash video analysis is a challenging multitask problem due to the complex spatiotemporal dynamics of crash events in video data and the diverse analytical requirements involved. It requires capabilities spanning crash recognition, temporal grounding, and high-level video understanding. Existing models, however, cannot perform all these tasks within a unified framework, and effective training strategies for such models remain underexplored. To fill these gaps, this paper proposes CrashChat, a multimodal large language model (MLLM) for multitask traffic crash analysis, built upon VideoLLaMA3. CrashChat acquires domain-specific knowledge through instruction fine-tuning and employs a novel multitask learning strategy based on task decoupling and grouping, which maximizes the benefit of joint learning within and across task groups while mitigating negative transfer. Numerical experiments on consolidated public datasets demonstrate that CrashChat consistently outperforms existing MLLMs across model scales and traditional vision-based methods, achieving state-of-the-art performance. It reaches near-perfect accuracy in crash recognition, a 176\% improvement in crash localization, and a 40\% improvement in the more challenging pre-crash localization. Compared to general MLLMs, it substantially enhances textual accuracy and content coverage in crash description and reasoning tasks, with 0.18-0.41 increases in BLEU scores and 0.18-0.42 increases in ROUGE scores. Beyond its strong performance, CrashChat is a convenient, end-to-end analytical tool ready for practical implementation. The dataset and implementation code for CrashChat are available at https://github.com/Liangkd/CrashChat.

多模态事故分析视频理解大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。