提出新指标衡量仿真与真实图像的决策一致性,解决'看起来像'却不能用的问题。
Quantifying Fidelity: A Decisive Feature Approach to Comparing Synthetic and Real Imagery
- 通过可解释AI识别模型决策的关键特征并比对
- 在2126组数据上发现传统方法忽略的差异
- 适合自动驾驶系统验证与仿真器优化人员
使用合成数据进行虚拟测试已成为自动驾驶安全验证的核心。尽管先进模拟器和生成式AI提升了视觉逼真度,但最新研究显示,仅像素级相似无法保证从仿真到现实的可靠迁移。真正关键在于系统在真实与仿真环境中是否基于一致的决策依据做出判断。为此,本文提出行为基础的保真度度量——决定性特征保真度(DFF),一种针对具体系统(SUT)的新指标,扩展了现有保真度谱系以捕捉机制一致性,即模型特定的决定性证据在不同域间的匹配程度。DFF利用可解释AI方法识别并比对驱动SUT输出的决定性特征。我们进一步提出基于反事实解释的估计器,并设计了DFF引导的校准方案以提升模拟器保真度。在2126对匹配的KITTI-VirtualKITTI2数据上的实验表明,DFF揭示了传统输出值保真度所忽视的差异。结果还显示,DFF引导校准在不牺牲输出值保真度的前提下,同时提升了决定性特征和输入层保真度,适用于多种SUT。
原文摘要 · Abstract (English)
Virtual testing using synthetic data has become a cornerstone of autonomous vehicle (AV) safety assurance. Despite progress in improving visual realism through advanced simulators and generative AI, recent studies reveal that pixel-level fidelity alone does not ensure reliable transfer from simulation to the real world. What truly matters is whether the system-under-test (SUT) bases its decisions on consistent decision evidence in both real and simulated environments, not just whether images "look real" to humans. To this end this paper proposes a behavior-grounded fidelity measure by introducing Decisive Feature Fidelity (DFF), a new SUT-specific metric that extends the existing fidelity spectrum to capture mechanism parity, that is, agreement in the model-specific decisive evidence that drives the SUT's decisions across domains. DFF leverages explainable-AI methods to identify and compare the decisive features driving the SUT's outputs for matched real-synthetic pairs. We further propose estimators based on counterfactual explanations, along with a DFF-guided calibration scheme to enhance simulator fidelity. Experiments on 2126 matched KITTI-VirtualKITTI2 pairs demonstrate that DFF reveals discrepancies overlooked by conventional output-value fidelity. Furthermore, results show that DFF-guided calibration improves decisive-feature and input-level fidelity without sacrificing output value fidelity across diverse SUTs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。