用搜索算法生成汽车场景，高效发现视觉语言模型的缺陷。
Search-based Testing of Vision Language Models for In-Car Scene Understanding
- 将测试建模为优化问题，自动调整场景参数生成多样化测试用例。
- 相比随机生成，故障率提升10倍，故障覆盖率提高3.6倍。
- 适合自动驾驶系统验证人员和视觉语言模型开发者使用。
在汽车领域，车内场景理解（ISU）可检测驾驶员分心等安全事件，并通过分析车内场景调节环境（如灯光）。行业正探索视觉语言模型（VLMs）来解析车载摄像头记录的场景并支持下游推理任务。然而，VLMs可能生成不完整、错误或误导性的场景描述，亟需系统化测试。真实车辆数据采集成本高、难扩展，尤其在早期设计阶段难以实现。本文提出ISU-Test，一种结合渲染场景生成与搜索式测试的自动化方法，将测试视为优化问题，系统性修改场景参数以生成多样化的车内场景并探索广泛配置。我们在工业原型和开源VLM上进行两项案例研究：问答与图文生成，对比随机场景生成基线。结果表明，ISU-Test显著优于基线，故障率最高提升10倍，故障覆盖率最高提升3.6倍。
原文摘要 · Abstract (English)
In the automotive domain, in-car scene understanding (ISU) enables the detection of safety-critical events, such as driver distraction, and supports drivers or passengers by analyzing the in-car scene and adapting the environment (e.g., ambient lighting). The industry is increasingly exploring vision-language models (VLMs) to interpret camera-recorded in-car scenes and extract information for downstream reasoning tasks. However, VLMs may generate incomplete, erroneous, or misleading scene descriptions, highlighting the need for systematic testing. Collecting real in-vehicle data is costly, difficult to scale, and often infeasible, particularly in early design stages. In this paper, we present ISU-Test, an automated testing approach that combines rendering-based scene generation with search-based testing to evaluate ISU systems. By framing testing as an optimization problem and systematically modifying scene parameters, our method generates diverse in-car scenarios and explores a wide range of configurations. We evaluate ISU-Test on both an industrial prototype and open-source VLMs across two case studies: question answering and captioning, comparing against randomized scenario generation. Results show that ISU-Test significantly outperforms the baseline, achieving up to 10 times higher failure rates and up to 3.6 times higher failure coverage.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。