对比解释方法在传统与智能体AI中的效果，发现轨迹分析更有效。
From Features to Actions: Explainability in Traditional and Agentic AI Systems
- 用特征归因和执行轨迹诊断对比解释效果
- 状态跟踪不一致导致失败概率降低49%，出现频率高2.7倍
- 智能体系统需转向轨迹级解释，而非单步输出分析
过去十年，可解释AI主要聚焦于单次预测的解释，生成基于输入输出的静态归因。近年来大语言模型（LLMs）推动了多步决策的智能体系统发展，其成功与否取决于决策序列而非单一输出。现有静态解释方法能否适用于动态行为尚不明确。本文通过对比归因方法与轨迹诊断，在两类场景中评估解释能力。结果表明，归因方法在静态设置中表现稳定（斯皮尔曼相关系数ρ = 0.86），但无法可靠诊断智能体轨迹中的执行失败。相反，基于轨迹的评分体系能持续定位行为崩溃点，揭示状态跟踪不一致在失败运行中出现频率高出2.7倍，且使成功概率下降49%。研究支持向轨迹级可解释性转型，以评估和诊断自主智能体行为。代码：https://github.com/VectorInstitute/unified-xai-evaluation-framework 项目页：https://vectorinstitute.github.io/unified-xai-evaluation-framework
原文摘要 · Abstract (English)
Over the last decade, Explainable AI has primarily focused on interpreting individual model predictions, producing post-hoc explanations that relate inputs to outputs under a fixed decision structure. Recent advances in large language models (LLMs) have enabled agentic AI systems whose behaviour unfolds over multi-step trajectories. In these settings, success and failure are determined by sequences of decisions rather than a single output. It remains unclear how explanation approaches designed for static predictions translate to agentic settings where behaviour emerges over time. In this work, we bridge this gap by comparing attribution-based explanations with trace-based diagnostics across both settings. Our results show that while attribution methods achieve stable feature rankings in static settings (Spearman \r{ho} = 0.86), they cannot be applied reliably to diagnose execution-level failures in agentic trajectories. In contrast, trace-grounded rubric evaluation for agentic settings consistently localizes behaviour breakdowns and reveals that state tracking inconsistency is 2.7x more prevalent in failed runs and reduces success probability by 49%. These findings motivate a shift towards trajectory-level explainability for evaluating and diagnosing autonomous AI behaviour in agentic systems. Code: https://github.com/VectorInstitute/unified-xai-evaluation-framework Project page: https://vectorinstitute.github.io/unified-xai-evaluation-framework
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。