arXiv:2606.08275cs.LGcs.AI2026-06被引 6

用因果干预定位大模型智能体失败的真正原因。

Causal Agent Replay: Counterfactual Attribution for LLM-Agent Failures

论文配图:Causal Agent Replay: Counterfactual Attribution for LLM-Agent Failures
图 1 · 摘自论文原文
  • 将智能体运行建模为因果图,通过干预步骤并重演轨迹来追踪失败根源。
  • 在合成数据上精准识别关键步骤和双步交互,准确率接近理论值(0.91)。
  • 适合关注AI系统可解释性与故障排查的研究者和工程师。

当大语言模型智能体出错——如错误退款、调用错误工具或泄露数据时,现有工具只能说明发生了什么(可观测性)或是否通过(评估),却无法判断是哪一步导致失败。常见启发式方法不可靠:执行有害操作的步骤通常并非决策步骤;而依赖大模型判断的归因方法相关性强但不可靠(当前最先进方法在Who&When基准上准确率仅约14%)。本文提出因果智能体重播(Causal Agent Replay, CAR),通过干预方式回答问题:将智能体运行建模为结构因果模型,对某一步施加do-操作,再在相同随机策略下向前重演轨迹,测量结果分布的变化。我们定义了针对智能体步骤的干预代数,设计单步对比估计器,其‘提交点’规则解决了随机前向执行中的混杂问题,并提出预算受限的蒙特卡洛沙普利估计器,用于分解相互作用步骤间的责任分配。所有影响均附置信区间。在带预设真实答案的合成因果模型上验证:对比估计器能恢复关键步骤,沙普利值准确捕捉两步交互(0.44, 0.45, ~0;效率和0.909,接近解析解0.91)。CAR开源,可在托管或免费本地模型上运行。

原文摘要 · Abstract (English)

When an LLM agent fails -- issues a refund it should not have, calls the wrong tool, leaks data -- existing tooling answers what happened (observability) or whether it passed (evaluation), but not which step caused the failure. The obvious heuristics are wrong: the step that executes the harmful action is usually not the step that decided on it, and LLM-judge attribution is correlational and unreliable (state-of-the-art step-level accuracy on the Who&When benchmark is about 14%). We present Causal Agent Replay (CAR), which answers the question by intervention: it models an agent run as a structural causal model, applies a do-operation to a step, and re-executes the trajectory forward under the same stochastic policy, measuring the shift in the outcome distribution. We define an intervention algebra over agent steps, a single-step contrastive estimator whose point-of-commitment rule resolves a confound specific to stochastic run-forward, and a budget-bounded Monte-Carlo Shapley estimator that splits credit across interacting steps. Every effect is reported with confidence intervals. We validate against synthetic structural causal models with planted ground truth: the contrastive estimator recovers the pivotal step, and Shapley recovers a two-step interaction (0.44, 0.45, ~0; efficiency sum 0.909 versus the analytic 0.91). CAR is open source and runs on hosted or free local models.

因果推理大模型可解释性故障诊断

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。