用真实事故规程增强视觉语言模型，自动判断是否该派救护车。
DispatchRAG: Grounding Emergency Dispatch Decisions in Real-World Protocols from Traffic Accident Video
- 基于检索增强生成，从日本事故规程中找最相关条目
- 在真实数据集上表现优于基础VLM，准确识别需派救护车场景
- 适合自动驾驶系统用于事故自动上报与应急决策
评估交通事故严重程度对决定派遣何种应急服务至关重要。错失对行人事故的救护车派遣可能导致死亡。尽管视觉语言模型（VLMs）在事故推理方面展现出潜力，但许多模型未基于真实事故应对规程，无法直接用于严重性评估。我们提出DispatchRAG框架，基于日本交通事故应对规程，增强VLM以生成恰当的应急响应。该框架采用基于RAG的检索机制获取最相关的事故规程，并通过大语言模型驱动的推理器建议最优响应。为支持评估，我们构建了事故调度数据集（Accident Dispatch Dataset），该数据集根据日本事故规程改编自MM-AU数据集，涵盖事故评估与应急响应。我们在该数据集上验证框架，结果显示其在多种事故场景下性能显著优于基线VLM，具备集成于自动驾驶系统中自动报告自身及周边事故的潜力。
原文摘要 · Abstract (English)
Assessing the severity of a traffic accident scenario is important to decide which emergency service to dispatch. Missing an ambulance dispatch on a pedestrian accident is a fatal issue that can lead to death. Recently, Vision-Language Models (VLMs) have been a promising tool for accident reasoning, yet many VLMs are not grounded in real-life accident response protocols, making them not usable in accident severity assessment off-the-shelf. We introduced DispatchRAG, an accident assessor and dispatcher framework grounded in real-life Japanese traffic-accident response protocols, designed to enhance VLMs to generate an appropriate emergency response during an emergency scenario. Utilizing a RAG-based retrieval mechanism to retrieve the most relevant accident protocol and an LLM-powered reasoner to suggest the most proper response. To support evaluation, we introduce Accident Dispatch Dataset, a comprehensive dataset of accident assessment and emergency response according to Japanese accident response protocols adapted from the MM-AU dataset. We validate our framework on the Accident Dispatch Dataset, showing strong performance across various accident scenarios compared to the baseline VLM, pointing toward integration in autonomous vehicles that can automatically report both their own and nearby accidents.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。