arXiv:2605.23899cs.AI2026-05被引 14

研究模型生成技能的全生命周期,发现其效果因人而异且易产生负迁移。

From Raw Experience to Skill Consumption: A Systematic Study of Model-Generated Agent Skills

论文配图:From Raw Experience to Skill Consumption: A Systematic Study of Model-Generated Agent Skills
图 1 · 摘自论文原文
  • 构建评估框架,系统测试技能生成、提取与使用全过程。
  • 模型生成技能平均有效,但存在显著负迁移,性能不随规模提升。
  • 提出元技能机制,可跨领域提升技能质量并减少负面效应。

语言代理通过复用从过往经验中提炼出的结构化技能不断改进。其中,领域级和模型生成的技能尤为有潜力:它们能快速适应特定领域,编码重复性操作流程,并突破人工设计的局限。然而,尽管提取方法日益增多,对技能全生命周期——经历生成、技能提取、技能消费——的理解仍不充分,尚不清楚这些技能是否真正有效、在何种情况下有效,以及成败原因。为此,我们建立基于效用的评估框架,覆盖五个多样化智能体任务领域,系统对比多种提取器与目标代理。结果表明,模型生成技能平均有益,但存在非平凡的负迁移;不同模型在提取与消费能力上表现不一,技能效用与模型规模或基础任务强度无关。进一步深入分析各阶段发现,经历构成影响技能质量,有效技能具有特定属性,同一技能在不同消费者间转移效果差异显著。最终,我们提炼出一种元技能,指导提取过程聚焦于实际效用相关特征,该策略在多个领域均显著提升技能质量并大幅降低负迁移。

原文摘要 · Abstract (English)

Language agents increasingly improve by reusing \emph{skills} -- structured procedural artifacts distilled from past experience. In particular, \emph{domain-level} and \emph{model-generated} skills are especially promising. They offer fast adaptation within a domain by encoding domain-specific recurring procedures, and they scale beyond labor-intensive hand-crafting. However, while extraction methods continue to proliferate, understanding remains limited, with no comprehensive study spanning the full skill lifecycle -- \textbf{experience generation}, \textbf{skill extraction}, and \textbf{skill consumption} -- to ask whether such skills actually work, when they work, and what makes them succeed or fail. To close this gap, we build a utility-grounded evaluation framework that provides systematic experimental results across extractors and target agents, covering five diverse agentic task domains. We find that model-generated skills are beneficial on average but exhibit non-trivial negative transfer, and that neither extractors nor targets behave uniformly. A model can be a strong extractor yet a weak consumer, or vice versa, with skill utility independent of model scale or baseline task strength. To explain these patterns, we then dissect each lifecycle stage in depth, analyzing how experience composition shapes skill quality, what properties characterize useful skills, and how the same skill transfers across different consumers. Finally, we translate these findings into a concrete \emph{meta-skill} that guides skill extraction toward the features tied to actual utility, which consistently improves skill quality across domains and substantially reduces negative transfer.

智能体技能提取负迁移元技能

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。