arXiv:2604.21255cs.CL2026-04ACL被引 1

量化工具使用行为的同质化程度,揭示大模型代理的模仿现象。

When Agents Look the Same: Quantifying Distillation-Induced Similarity in Tool-Use Behaviors

论文配图:When Agents Look the Same: Quantifying Distillation-Induced Similarity in Tool-Use Behaviors
图 1 · 摘自论文原文
  • 用图结构分析工具调用习惯,分离出非必要行为模式。
  • 同一厂商模型间工具行为相似度高出5.9个百分点。
  • 适合研究模型演化、安全对齐与多样性评估的学者。

模型蒸馏推动了大语言模型代理的快速发展,但常导致行为同质化。众多新兴代理表现出几乎相同的推理步骤和失败模式,暗示其可能源自少数主导教师模型的复刻。现有度量方法无法区分任务成功必需行为与反映自主偏好的非必需模式。为此,我们提出两个互补度量:响应模式相似性(RPS)用于衡量语言层面的对齐,动作图相似性(AGS)将工具使用习惯建模为有向图以捕捉行为特征。在τ-Bench和τ²-Bench上评估18个来自8家厂商的模型,以Claude Sonnet 4.5(思考模式)为基准,发现同家族模型对的AGS得分比跨家族模型对高5.9个百分点;Kimi-K2(思考模式)达到82.6%的节点相似度(S_node)和94.7%的依赖关系相似度(S_dep),超过Anthropic自有的Opus 4.1。受控蒸馏实验进一步验证,AGS可区分教师特异性趋同与普遍性能提升。RPS与AGS捕捉不同行为维度(皮尔逊相关系数r = 0.491),为代理生态中的行为收敛提供互补诊断信号。代码已开源于https://github.com/Syuchin/AgentEcho。

原文摘要 · Abstract (English)

Model distillation is a primary driver behind the rapid progress of LLM agents, yet it often leads to behavioral homogenization. Many emerging agents share nearly identical reasoning steps and failure modes, suggesting they may be distilled echoes of a few dominant teachers. Existing metrics, however, fail to distinguish mandatory behaviors required for task success from non-mandatory patterns that reflect a model's autonomous preferences. We propose two complementary metrics to isolate non-mandatory behavioral patterns: \textbf{Response Pattern Similarity (RPS)} for verbal alignment and \textbf{Action Graph Similarity (AGS)} for tool-use habits modeled as directed graphs. Evaluating 18 models from 8 providers on $τ$-Bench and $τ^2$-Bench against Claude Sonnet 4.5 (thinking), we find that within-family model pairs score 5.9 pp higher in AGS than cross-family pairs, and that Kimi-K2 (thinking) reaches 82.6\% $S_{\text{node}}$ and 94.7\% $S_{\text{dep}}$, exceeding Anthropic's own Opus 4.1. A controlled distillation experiment further confirms that AGS distinguishes teacher-specific convergence from general improvement. RPS and AGS capture distinct behavioral dimensions (Pearson $r$ = 0.491), providing complementary diagnostic signals for behavioral convergence in the agent ecosystem. Our code is available at https://github.com/Syuchin/AgentEcho.

模型蒸馏行为同质化代理评估

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。