研究视觉语言模型在自动驾驶中对危险检测的稳定性表现
Task-Aligned Stability Analysis of Vision-Language Models for Autonomous Driving Hazard Detection
- 用受控干扰测试图像嵌入与危险评分的关联性
- 部分干扰导致嵌入微变但危险判断大幅失准
- 不同干扰类型引发不同类型错误，需针对性评估
视觉语言模型（VLMs）在自动驾驶场景理解中应用日益广泛，但现有鲁棒性分析多依赖任务无关的嵌入稳定性。本文研究噪声引起的嵌入漂移是否能预测基于CLIP图像-文本相似度的任务对齐危险评分变化。在BDD100K道路场景上施加受控干扰，对比嵌入漂移与危险评分边际漂移（即扰动下危险得分的变化）。结果表明该关系高度依赖干扰类型：某些干扰家族中表示漂移与决策漂移强相关，而另一些则在嵌入变化较小的情况下引发严重决策不稳。此外，干扰类型在失败方向上存在差异：多数导致假阴性（抑制危险检测），而遮挡反而引发假阳性（误报危险）。这表明基准设计应考虑非对称失败模式，而不仅关注整体不稳定性。研究建议鲁棒性评估应包含任务对齐的稳定性指标，而非仅依赖嵌入级扰动统计。
原文摘要 · Abstract (English)
Vision-language models (VLMs) are increasingly used for scene understanding in autonomous driving, but robustness analysis often relies on task-agnostic embedding stability alone. We study whether corruption-induced embedding drift predicts changes in a task-aligned hazard score derived from CLIP image-text similarities. Using controlled corruptions on BDD100K road scenes, we compare embedding drift against margin drift, defined as the change in hazard score under perturbation. The relationship is highly corruption-dependent: some families exhibit strong coupling between representation drift and decision drift, while others induce hazardous decision instability despite relatively modest embedding change. Furthermore, corruption families differ in failure direction: most suppress hazard detections via false negatives, while occlusion instead triggers false alarms, suggesting that benchmark design should account for asymmetric failure modes, not just overall instability rates. These results suggest that robustness benchmarks should include task-aligned stability measures in addition to embedding-level perturbation statistics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。