用大模型生成真实道路故障场景,测试自动驾驶边缘系统的鲁棒性。
LLM-Generated Fault Scenarios for Evaluating Perception-Driven Lane Following in Autonomous Edge Systems
- 用LLM生成语义故障场景,LDM合成高保真传感器退化。
- 在460种故障下,定位精度下降至31.0%,误差飙升99%。
- 适合做边缘AI安全验证的开发者和自动驾驶测试团队。
将自主视觉系统部署于边缘设备面临资源限制,难以实时执行全面安全测试。现有验证方法依赖静态数据集或人工注入故障,无法覆盖真实环境中的多样风险。为此,我们提出解耦的离线-在线故障注入框架:离线阶段利用大语言模型(LLM)生成结构化故障场景,结合潜在扩散模型(LDM)合成高保真传感器退化;这些复杂故障动态被压缩为预计算查表,供边缘设备实时进行带故障感知的推理,无需本地运行重型AI模型。我们在ResNet18车道保持模型上验证该框架,共涵盖460个故障场景。结果显示,模型在干净数据下的基准R²约为0.85,但在生成故障下表现出显著鲁棒性下降——雾天条件下,均方根误差最高上升99%,定位精度(within-0.10)最低降至31.0%,表明仅基于正常数据评估不足以支撑真实边缘AI部署。
原文摘要 · Abstract (English)
Deploying autonomous vision systems on edge devices faces a critical challenge: resource constraints prevent real-time and predictable execution of comprehensive safety tests. Existing validation methods depend on static datasets or manual fault injection, failing to capture the diverse environmental hazards encountered in real-world deployment. To address this, we introduce a decoupled offline-online fault injection framework. This architecture separates the validation process into two distinct phases: a computationally intensive Offline Phase and a lightweight Online Phase. In the offline phase, we employ Large Language Models (LLMs) to semantically generate structured fault scenarios and Latent Diffusion Models (LDMs) to synthesize high-fidelity sensor degradations. These complex fault dynamics are distilled into a pre-computed lookup table, enabling the edge device to perform real-time fault-aware inference without running heavy AI models locally. We extensively validated this framework on a ResNet18 lane-following model across 460 fault scenarios. Results show that while the model achieves a baseline R^2 of approximately 0.85 on clean data, our generated faults expose significant robustness degradation, with RMSE increasing by up to 99% and within-0.10 localization accuracy dropping to as low as 31.0% under fog conditions, demonstrating the inadequacy of normal-data evaluation for real-world edge AI deployment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。