arXiv:2607.22520cs.AI2026-07

给大模型加技能反而让任务失败?研究揭示了三大原因。

The Regression Tax: Decomposing Why Skills Help and Hurt LLM Agents

论文配图:The Regression Tax: Decomposing Why Skills Help and Hurt LLM Agents
图 1 · 摘自论文原文
  • 通过6000次实验对比有无技能的表现,发现技能会引发新失败
  • 最佳技能优势来自减少退化而非提升性能,退化占主导
  • 关键在加强输入理解和输出检查,而非堆砌流程指令

为大模型代理添加程序性技能通常以任务成功率平均提升来评估,但这一指标掩盖了重要代价:技能可能使代理表现更差。我们在近6000次运行中,对比了两个办公自动化基准和三个模型测试框架下有无技能的表现。结果区分出两类现象:回归(无技能时成功,加技能后失败)与残余失败(无论有无技能均失败)。发现回归现象显著,最佳技能的优势主要源于较少退化,而非更多增益。识别出三种退化成因:(i) 技能描述渗透,即技能存在本身改变代理行为,即使未被调用;(ii) 基于定位偏移,技能规程覆盖代理对输入的解释;(iii) 验证抑制,规程压制了代理本应执行的输出检查。分析持续失败模式显示,现有技能过度强调流程指导(失败率最低阶段),而忽视了定位与验证(错误主因)。纠正评估偏差并分析执行轨迹后,发现多数退化与持续失败可通过改进定位与验证恢复。建议将技能评估分解为增益与退化两部分,而非仅看整体提升。我们识别出三种应避免的退化模式,表明可靠性更依赖定位与验证,而非技能选择本身。

原文摘要 · Abstract (English)

Adding procedural skills to an LLM agent is typically evaluated by average improvement in task success. However, this metric hides an important cost: skills can also make agents worse. We measure both sides by comparing agents with and without skills across nearly 6,000 runs spanning two office automation benchmarks and three model harness stacks. This allows us to distinguish two outcomes. A regression is a task solved without skills but failed after skills are added. A residual failure is a task that fails both with and without skills. We find that regressions are substantial enough that the best performing skills outperform others primarily by regressing less, not by gaining more. We identify three causes of regression: (i) skill description osmosis, a skill changes an agent's behavior simply by being present in context, even when it is never invoked; (ii) grounding displacement, a skill's prescribed procedure overrides how the agent interprets its inputs; and (iii) verification displacement, where the procedure suppresses checks the agent would otherwise perform on its outputs. Analysing persistent failures reveals the same underlying pattern. Existing skills overemphasize procedural guidance the stage least often responsible for failure while under supporting grounding and verification, the dominant sources of remaining errors. After correcting evaluation artifacts and studying traces, we find many regressions and persistent failures recoverable through better grounding and verification. Procedural skills should be evaluated by decomposing their net effect into gains and regressions, not by aggregate improvement alone. We identify three regression modes skills should avoid, and find that reliability depends more on grounding and verification than on procedural skill choice.

大模型技能退化可靠性评估方法

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。