arXiv:2606.14239cs.AI2026-06被引 7

无需真实标签,通过对比行为自动优化智能体技能。

SkillAudit: Ground-Truth-Free Skill Evolution via Paired Trajectory Auditing

论文配图:SkillAudit: Ground-Truth-Free Skill Evolution via Paired Trajectory Auditing
图 1 · 摘自论文原文
  • 用相同任务有无技能的对比,分析技能对行为的影响。
  • 在89个任务中平均得分73.9%,超越无技能代理(40.9%)。
  • 适合缺乏标注数据的技能持续优化场景。

智能体技能是引导冻结大模型执行特定流程的结构化指令包。部署后技能常因边缘情况、API变更或环境约束而失效,因此技能演化成为必要。现有方法依赖隐藏反馈如验证分数、测试结果或环境奖励,但这些信号在仅有任务描述和工作区数据时往往不可得。本文提出SkillAudit,一种无需真实标签的技能演化框架。核心思路是成对轨迹审计:每次迭代中,同一任务分别在有无候选技能下执行,隔离技能对行为的影响。为将行为差异转化为修改指引,采用过程对齐对比评估(PACE),一组评估器将轨迹差异映射到技能文档中具体段落的诊断信号。一个基于任务规范编译的结构化验证器,可检查任务约束并回滚有害更新。编辑通过两条路径进行:Refine 去除通用技能中的噪声指引,Repair 替换与任务冲突的段落。在覆盖8个专业领域的89个容器化任务中,SkillAudit实现平均任务奖励73.9%,优于无技能代理(40.9%)和静态专家技能(56.7%),且演化过程中未访问隐藏测试、参考解或外部评分函数。

原文摘要 · Abstract (English)

Agent skills are structured procedural packages that guide frozen LLM agents in specialized workflows. Skills rarely remain sufficient after deployment: edge cases, API changes, and deployment constraints become visible only through use, making skill evolution a practical necessity. Existing methods depend on privileged feedback such as held-out validation scores, hidden test outcomes, or environment rewards -- signals often unavailable when a practitioner has only a task description and workspace data. We introduce SkillAudit, a framework for evolving agent skills without ground-truth feedback. The key idea is paired trajectory auditing: at each iteration, the same task is executed with and without the candidate skill, isolating how the skill changes agent behavior without external labels. To turn behavioral differences into edit guidance, SkillAudit uses Process-Aligned Contrastive Evaluation (PACE), a cluster of evaluators that maps trajectory divergences to diagnostic signals linked to specific passages in the skill document. A structural verifier, compiled once from the task specification and then fixed, checks task constraints and rolls back harmful updates. SkillAudit routes edits through two pipelines: Refine removes noisy or irrelevant guidance from broadly useful skills, while Repair replaces passages that conflict with the task. Across 89 containerized tasks spanning 8 professional domains, SkillAudit achieves 73.9% average task reward, outperforming an agent without skills (40.9%) and the static expert skill (56.7%). These gains are obtained without accessing hidden tests, reference solutions, or external scoring functions during evolution.

智能体技能演化无监督学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。