无需额外计算，通过路由权重直接生成多层级AI系统归因图谱。
BOHM: Zero-Cost Hierarchical Attribution for Compound AI Systems
- 基于系统已有路由权重构建层次化归因树，不需访问组件内部。
- 在18个LLM的3层结构中，仅用少量评估即达到0.928的Kendall tau相关性。
- 适合无法访问的API或代理系统，可诊断与传统方法的差异。
复合AI系统通过专业化组件的层级结构处理任务。归因分析长期依赖基于Shapley值的方法（SHAP），该方法需对任意组件子集进行系统评估，但在第三方API、黑盒接口和代理编排器中难以实现，因其路由高度集中于少数工具，导致多数组合无法评估。本文提出BOHM，直接从系统已维护的路由权重中提取层次化归因树：叶节点归因为根到叶路径权重乘积；第k层归因为深度为k节点的分布。该方法零边际成本，无需组件内核访问，且能同时提供各层级的多分辨率归因，这是平坦型方法在任何评估预算下都无法做到的。BOHM与SHAP回答不同问题，当路由器近似最优时二者趋于一致。在880个LiveCodeBench问题上，18个大模型构成3层结构中，BOHM的Kendall tau达0.928；而SHAP需9000倍更多组合评估才达0.980。在5个驱动器、7个基准测试的完整覆盖实验中（35个单元），驱动器普遍集中使用单一工具（最高占比中位数0.65），单元级tau(BOHM, SHAP)由驱动器首选是否为实际最优工具决定（均值+0.22对比~+0.01）。在475个叶子节点、4层结构的美国人口普查数据中，BOHM在各层级均恢复真实排序（tau最高0.722）。BOHM满足效率、单调性、对称性和弱抑制性，但不满足Shapley的可加性。其本质是互补基础工具：一种可在存在路由状态处计算的多分辨率分解，与Shapley的差异本身具有诊断意义。
原文摘要 · Abstract (English)
Compound AI systems route tasks through hierarchies of specialised components. Attribution is dominated by Shapley-based methods (SHAP), which decompose a coalition value function into per-component marginal contributions and require evaluation of the system on arbitrary component subsets. That requirement fails for third-party APIs, opaque endpoints, and agentic orchestrators that concentrate routing on a few tools, leaving most coalitions un-evaluable from the deployed orchestrator. We introduce BOHM, which extracts a hierarchical attribution tree directly from the routing weights such systems already maintain: leaf attribution is the path product of root-to-leaf routing weights; level-k attribution is the induced distribution over depth-k nodes. The method has zero marginal cost, requires no access to component internals, and provides multi-resolution attribution at every level simultaneously, which flat methods cannot offer at any evaluation budget. BOHM and SHAP answer different questions and converge when the deployed router routes near-optimally. On 18 LLMs in a 3-level hierarchy over 880 LiveCodeBench problems, BOHM yields Kendall tau=0.928; SHAP reaches tau=0.980 at 9,000x more coalition evaluations per seed. On a 5-driver, 7-benchmark agentic study (35 cells, complete coverage), drivers concentrate routing on a single tool (top-share median 0.65), and cell-level tau(BOHM,SHAP) is predicted by whether the driver's top pick is the empirically best tool (mean +0.22 vs ~+0.01). On a US Census hierarchy (475 leaves, 4 levels), BOHM recovers ground-truth rankings at every level (tau up to 0.722). BOHM satisfies efficiency, monotonicity, symmetry, and weak suppression but not Shapley's additivity. It is best understood as a complementary primitive: a multi-resolution decomposition computable wherever routing state exists, whose disagreement with Shapley is itself diagnostic.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。