arXiv:2605.27879cs.AI2026-05

提出验证机制与开放世界基准,提升大模型解释的可信度

Towards Faithful Agentic XAI: A Verification Method and an Open-World Benchmark for Better Model Faithfulness

论文配图:Towards Faithful Agentic XAI: A Verification Method and an Open-World Benchmark for Better Model Faithfulness
图 1 · 摘自论文原文
  • 将解释拆分为待验证陈述,用可靠工具逐条核对
  • 在新基准上使解释可信度从0.20提升至0.46,且保持表达质量
  • 适合关注模型解释真实性的研究人员和实践者

可解释人工智能(XAI)帮助用户理解模型行为并发现潜在错误。基于大语言模型(LLM)的智能体式XAI系统虽提升了交互自然性,但可能生成看似合理却失真的解释。此类风险源于复杂模型的不可靠输出被LLM放大,误导用户。本文提出忠实智能体式XAI(FAX)框架,通过显式验证提升解释可信度:将草稿解释分解为若干陈述,交叉核对其与内在可信工具的一致性,过滤不支持或矛盾的陈述后再生成最终结果。同时引入CRAFTER-XAI-Bench,一个包含复杂策略、多样目标和挑战场景的开放世界强化学习基准,用于评估模型特定的解释可信度。在该基准上,FAX将模拟可信度从最强基线的0.20提升至0.46,同时保持高信息量、相关性和流畅性。在三个表格基准上,FAX表现媲美现有智能体式XAI基线,但分析表明这些设置易将任务准确率与模型专属可信度混淆。研究证明,显式验证对实现可信智能体式XAI至关重要,且可信度评估基准必须针对目标模型的行为设计。

原文摘要 · Abstract (English)

Explainable AI (XAI) helps users interpret model behavior and identify potential faults. Agentic XAI systems use Large Language Models (LLMs) to make explanations more accessible through natural-language interaction, but they can also produce plausible yet unfaithful explanations. This risk arises because unreliable XAI outputs for complex models can be amplified by LLMs and mislead users. We propose Faithful Agentic XAI (FAX), a framework that improves explanation faithfulness through explicit verification. FAX decomposes draft explanations into claims and cross-checks them against inherently faithful tools, filtering unsupported or contradictory claims before final generation. We also introduce CRAFTER-XAI-Bench, an open-world reinforcement learning benchmark with complex policies, diverse goals, and challenging scenarios for assessing model-specific faithfulness. On CRAFTER-XAI-Bench, FAX improves simulation faithfulness from 0.20 for the strongest baseline to 0.46 while maintaining high informativeness, relevance, and fluency. On three tabular benchmarks, FAX performs competitively with prior Agentic XAI baselines, but our analysis shows that these settings can conflate task accuracy with model-specific faithfulness. These findings show that explicit verification is essential for faithful Agentic XAI and that that faithfulness benchmarks must be designed to test explanations against the behavior of the target model itself.

可解释AI大模型可信性验证强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。