用视觉语言模型实现动态观察后再报警,提升驾驶预警准确性。
Observe Before You Alert: Adaptive Driver Alerting with Vision-Language Models

- 引入观察-静默-报警三动作策略,延迟不确定警告
- 在真实数据集上达0.4878的部署效用,优于基线0.4752
- 适合需要自适应预警的智能驾驶系统研发
从行车记录仪视频中进行驾驶员预警需在部分可观测条件下做出序列决策:系统不仅要判断场景是否危险,还需确定证据充分时才发出警告。现有事故预测模型多输出二值风险评分,模糊场景依赖阈值处理。本文提出VLAlert框架,将报警生成建模为对SILENT、OBSERVE、ALERT三类动作的策略选择。其中OBSERVE作为内部证据收集决策,可延后不确定警告并调整下一次观测窗口,形成轻量级感知-行动闭环以实现自适应预警。该框架采用Qwen3-VL-4B生成安全证据,并通过结构化信念片段的隐藏状态池化,构建紧凑表征用于危险评估与策略预测。我们在四个真实世界行车记录仪预警数据集整合而成的VLAlert-Bench上评估,结果表明,在验证集上,VLAlert相较Open-BADAS达到更高的部署效用(DAUS 0.4878 vs. 0.4752),同时在AUROC、AP_tick、F1_t和平衡准确率上分别提升至0.689、0.195、0.297和0.648。在221个未见的ADAS接管测试片段中,其R@5s从74.2%提升至88.7%,F1从0.585升至0.686。结果表明,自适应观察机制与面向安全的视觉语言模型表征显著提升了驾驶员侧预警决策性能。
原文摘要 · Abstract (English)
Driver alerting from dashcam video requires sequential decision-making under partial observability: a system must decide not only whether a scene is risky, but also when the evidence is sufficient to warn. Most existing accident anticipation models output a binary risk score, leaving ambiguous scenes to be handled by thresholding. We propose VLAlert, a vision-language alerting framework that casts warning generation as a tri-action policy over SILENT, OBSERVE, and ALERT. The OBSERVE action acts as an internal evidence-gathering decision that delays uncertain warnings and changes the next observation window, creating a lightweight perception-action loop for adaptive alerting. VLAlert uses Qwen3-VL-4B as a safety-evidence generator and pools hidden states from structured belief spans to form compact representations for danger estimation and policy prediction. We evaluate VLAlert on VLAlert-Bench, a unified per-tick benchmark from four real-world dashcam alert datasets, and further test transfer to held-out naturalistic ADAS takeover clips. On VLAlert-Bench validation, VLAlert achieves the highest deployment-oriented utility among tested baselines, with DAUS 0.4878 compared with 0.4752 for Open-BADAS, and improves AUROC, AP_tick, F1_t, and balanced accuracy from 0.610, 0.176, 0.276, and 0.581 to 0.689, 0.195, 0.297, and 0.648, respectively. On 221 held-out ADAS-TO-Critic clips, VLAlert improves R@5s from 74.2% to 88.7% and F1 from 0.585 to 0.686. These results indicate that adaptive observation and safety-focused VLM representations provide measurable gains for driver-facing alert decisions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。