arXiv:2605.16087cs.ROcs.AI2026-05中稿 · publication at IEE…

让自动驾驶感知模型更可信:解释性+不确定性估计+实时部署

Towards Trustworthy and Explainable AI for Perception Models: From Concept to Prototype Vehicle Deployment

论文配图:Towards Trustworthy and Explainable AI for Perception Models: From Concept to Prototype Vehicle Deployment
图 1 · 摘自论文原文
  • 基于注意力机制生成可解释的检测结果,通过扰动测试验证其可靠性
  • 集成校准的不确定性估计,提升模型在极端场景下的鲁棒性
  • 已实现在原型车上的实时可信监控,适合自动驾驶安全研发人员

深度神经网络已成为自动驾驶感知的主流方案,但其黑箱特性与可信AI准则相悖,增加了安全验证、故障排查和人机监管的难度。尽管安全可解释AI(XAI)的理论框架存在,但在3D场景理解中的具体实现仍稀缺。本文提出一个可信感知模块,具备强鲁棒性、忠实的可解释性及校准的不确定性估计。基于Transformer检测器,在推理时利用注意力机制生成解释,并通过扰动一致性测试验证其真实性;进一步集成不确定性估计与校准模块,并采用增强鲁棒性的训练方法。实验表明,该模块具备忠实的显著性行为,提升鲁棒性,且不确定性估计准确。最终,我们将这些可信AI组件部署于原型车,提供可视化界面,实时展示文档、模型不确定性状态与显著图,证明了实时可信感知监控的可行性。补充材料见 https://tillbeemelmanns.github.io/trustworthy_ai/。

原文摘要 · Abstract (English)

Deep Neural Networks have become the dominant solution for Autonomous Driving perception, but their opacity conflicts with emerging Trustworthy AI guidelines and complicates safety assurance, debugging, and human oversight. While theoretical frameworks for safe and Explainable AI (XAI) exist, concrete implementations of Trustworthy AI for 3D scene understanding remain scarce. We address this gap by proposing a Trustworthy AI perception module that is remarkably robust, integrates faithful explainability, and calibrated uncertainty estimates. Building on a transformer-based detector, we derive explanation from the attention mechanism at inference time and validate their faithfulness using perturbation-based consistency tests. We further integrate an uncertainty estimation and calibration module, and apply robustness-enhancing training methods. Experiments show faithful saliency behavior, improved robustness, and well-calibrated uncertainty estimates. Finally, we deploy these Trustworthy AI elements in a prototype vehicle and provide an XAI Interface that visualizes documentation artifacts, model uncertainty state, and saliency maps, demonstrating the feasibility of trustworthy perception monitoring in real time. Supplementary materials are available at https://tillbeemelmanns.github.io/trustworthy_ai/ .

可信AI可解释性自动驾驶不确定性估计

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。