arXiv:2605.25310cs.CL2026-05

首次发现大模型在运行时能线性解码工具调用依赖结构。

Tool-Call Dependency Structure is Linearly Decodable in LLM Agent Residual Streams

  • 通过残差流中的低容量探测器,识别出工具调用的依赖图结构。
  • 在多跳任务中探测效果显著,单次规划时信号消失。
  • 信号反映抽象拓扑而非具体值,适用于交互式多步推理场景。

使用工具的大型语言模型代理会产生调用轨迹,其调用形成有向依赖图:早期工具输出为后续调用提供参数。这种执行结构是否存在于模型内部尚不清楚;以往的结构探测主要针对静态代码或思维链文本,而非代理的运行时调用图。对Qwen3-32B残差流进行的低容量边探测器,在远超Hewitt-Liang随机标签控制和位置基线的情况下,有效解码了工具调用依赖图。反事实对比显示,信号追踪的是抽象拓扑而非标识符值,并在独立非子串预言机下复现。非位置成分在三个额外的交互式多跳基准上重现,当仅靠调用顺序即可充分代理依赖时信号衰减,单次规划中完全消失。逐层激活修补实验表明,探测信号在未修补边界之后传播,证据显示该表示是动态传递而非被动读取,尽管实际调用内容未改变。据我们所知,这是首个针对大型语言模型代理运行时工具调用依赖图的结构探测。我们的结论涉及表征,而非行为控制,覆盖两个模型家族和一个主要领域。

原文摘要 · Abstract (English)

Tool-using LLM agents produce trajectories whose calls form a directed dependency graph: earlier tool outputs supply arguments to later calls. Whether this execution structure is represented inside the model is unknown; prior structural probes have targeted static code or chain-of-thought text, not an agent's run-time call graph. A low-capacity edge probe on the residual stream of Qwen3-32B decodes the tool-call dependency graph well above both a Hewitt--Liang random-label control and a positional baseline. A counterfactual contrast between value corruption and structural perturbation indicates the signal tracks abstract topology rather than identifier values, and replicates under an independent, non-substring oracle. The non-positional component replicates on three further interactive multi-hop benchmarks and attenuates as call order alone becomes a sufficient proxy for dependency, vanishing in single-shot planning. Per-layer activation patching shifts the probe at a later, non-patched boundary, evidence that the representation propagates rather than passively reads out, though the realised tool call does not move. To our knowledge this is the first structural probe of an LLM agent's runtime tool-call dependency graph. Our claims concern representation, not behavioural control, and span two model families and one primary domain.

大模型工具调用结构探针残差流

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。