让自动驾驶更可解释,提升人类对车辆行为的预判能力
Explainable deep learning improves human mental models of self-driving cars
- 用概念包装网络(CW-Net)将黑箱决策转化为人类可理解的原因
- 实车测试显示,解释信息显著提升驾驶员对车辆行为的预测准确率
- 方法适用于真实场景,对自动驾驶、无人机等安全系统有推广价值
自动驾驶汽车越来越多地依赖深度神经网络实现类人驾驶。这类黑箱规划器的不透明性使得人们难以准确预判其故障时刻,可能带来灾难性后果。尽管解释性研究迅速发展,但多数局限于仿真或简化环境,真实部署下的实用性仍不清楚。本文提出概念包装网络(CW-Net),一种能忠实解释基于机器学习规划器行为的方法,其推理基于人类可理解的概念,且不牺牲性能。我们在真实自动驾驶汽车上部署CW-Net,结果表明,生成的解释显著改善了驾驶员对车辆行为的心理模型,使其在意外情境下能更好预测车辆动作。这证明可解释深度学习在真实部署中既可理解又实用。本方法有望推广至其他高安全性系统,如自主无人机、机器人外科手术,以及端到端学习系统和视觉-语言-动作模型。总体而言,本研究为自主代理的可解释性提供了经实证验证的路径,有助于提升系统的透明度与安全性。
原文摘要 · Abstract (English)
Self-driving cars increasingly rely on deep neural networks to achieve human-like driving. The opacity of such black-box planners makes it challenging to accurately anticipate when they will fail, with potentially catastrophic consequences. While research into interpreting these systems has surged, most of it is confined to simulations or toy setups due to the difficulty of real-world deployment, leaving the practical utility of such techniques unknown. Here, we introduce the Concept-Wrapper Network (CW-Net), a method for faithfully explaining the behavior of machine-learning-based planners that causally grounds their reasoning in human-interpretable concepts without sacrificing performance. We deploy CW-Net on a real self-driving car and show that the resulting explanations improve the human driver's mental model of the vehicle, allowing them to better predict its behavior, particularly in surprising situations. This demonstrates that explainable deep learning integrated into self-driving cars can be both understandable and useful in a realistic deployment setting. We anticipate our method could be applied to other safety-critical systems, such as autonomous drones and robotic surgeons, as well as to other architectures, such as end-to-end learning systems and vision-language-action models. Overall, our study establishes a deployment-validated pathway to interpretability for autonomous agents, which could help make them more transparent and safe.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。