首次为大模型工具使用提供可解释性，判断每个工具的贡献度。
AgentSHAP: Interpreting LLM Agent Tool Importance with Monte Carlo Shapley Value Estimation
- 用蒙特卡洛采样估算博弈论中的谢尔比值，评估工具重要性
- 在API-Bank测试中准确区分关键与无关工具，结果稳定可靠
- 无需模型内部参数，适配所有主流大模型，通用性强
使用外部工具的大语言模型代理能解决复杂任务，但难以判断哪些工具真正贡献了结果。现有可解释性方法无法处理工具层面的解释。我们提出AgentSHAP，首个针对大模型代理工具重要性的解释框架。该方法不依赖模型结构，适用于任何大模型（如GPT、Claude、Llama），仅需将其视为黑盒。通过蒙特卡洛采样估算谢尔比值，测试代理在不同工具组合下的响应，基于博弈论计算公平的工具重要性评分。主要贡献：(1) 首个基于谢尔比值的代理工具归因解释方法；(2) 蒙特卡洛采样将计算复杂度从O(2n)降至实用水平；(3) 在API-Bank数据集上验证，结果跨多次运行一致，能正确识别关键工具并区分相关与无关工具。AgentSHAP与TokenSHAP（用于标记）和PixelSHAP（用于图像区域）共同构成生成式AI的谢尔比基可解释性工具家族。代码开源：https://github.com/GenAISHAP/TokenSHAP。
原文摘要 · Abstract (English)
LLM agents that use external tools can solve complex tasks, but understanding which tools actually contributed to a response remains a blind spot. No existing XAI methods address tool-level explanations. We introduce AgentSHAP, the first framework for explaining tool importance in LLM agents. AgentSHAP is model-agnostic: it treats the agent as a black box and works with any LLM (GPT, Claude, Llama, etc.) without needing access to internal weights or gradients. Using Monte Carlo Shapley values, AgentSHAP tests how an agent responds with different tool subsets and computes fair importance scores based on game theory. Our contributions are: (1) the first explainability method for agent tool attribution, grounded in Shapley values from game theory; (2) Monte Carlo sampling that reduces cost from O(2n) to practical levels; and (3) comprehensive experiments on API-Bank showing that AgentSHAP produces consistent scores across runs, correctly identifies which tools matter, and distinguishes relevant from irrelevant tools. AgentSHAP joins TokenSHAP (for tokens) and PixelSHAP (for image regions) to complete a family of Shapley-based XAI tools for modern generative AI. Code: https://github.com/GenAISHAP/TokenSHAP.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。