让大模型用假设驱动的方式安全优化外部技能,避免无效更新。
Hypothesis-Driven Skill Optimization for LLM Agents

- 基于可验证假设提出候选技能,通过对照实验筛选有效方案。
- 在ALFWorld上使Qwen3-8B提升6.9分,即使反馈错误率高达20%仍保持7.1分增益。
- 适合追求可解释性与安全性的冻结模型应用,尤其适合长期运行的智能体。
外部技能可在不修改模型权重的情况下提升面向行动的大语言模型智能体性能,但若从稀疏或噪声轨迹中提炼,则持续更新存在风险。一个合理的反思可能编码有效流程、虚假捷径,或执行器无法可靠遵循的规则。本文提出假设驱动的技能优化(HDSO),一种无需训练的框架,其中技能管理者与执行器均为冻结推理端点。管理者观察执行器轨迹,提出可证伪的假设并制定明确验证计划,将假设实例化为候选技能包,通过成对对照/处理执行验证,分析行为差异,并仅将被支持的候选加入已批准库。执行器通过渐进披露使用已批准技能,在无技能选择时仍保持仅执行器路径。在ALFWorld上,HDSO使Qwen3-8B基线平均成功率提升6.9点,Qwen3.6-27B提升4.0点。当技能发现与验证阶段存在20%随机翻转的成功/失败反馈时,仍保持对Qwen3-8B的+7.1点增益。跨任务与异构配对诊断表明,经验证的库可超越原生成任务使用,但跨模型管理仅在管理者诊断、执行器能力与验证证据一致时成功。HDSO为冻结行动智能体提供可审计的技能生命周期,而非无约束的记忆积累过程。
原文摘要 · Abstract (English)
External skills can improve action-oriented LLM agents without changing model weights, but persistent skill updates are risky when they are distilled from sparse or noisy trajectories. A plausible reflection may encode a useful procedure, a spurious shortcut, or a rule that the target executor cannot reliably follow. We propose Hypothesis-Driven Skill Optimization (HDSO), a train-free framework in which both the skill curator and the agent executor are frozen inference endpoints. The curator observes executor traces, proposes a falsifiable hypothesis with an explicit validation plan, instantiates the hypothesis as a candidate skill package, validates the package through paired control/treatment executions, reviews behavior differences, and consolidates only supported candidates into an approved repository. The executor consumes approved skills through progressive disclosure, preserving the executor-only path when no skill is selected. On ALFWorld, HDSO improves executor-only baselines by +6.9 Avg. SR points for Qwen3-8B and +4.0 points for Qwen3.6-27B. Under 20% randomly flipped success/failure feedback during skill discovery and validation, HDSO preserves a +7.1-point gain for Qwen3-8B. Transfer and heterogeneous-pair diagnostics further show that validated repositories can be useful beyond the run that produced them, but cross-model curation succeeds only when curator diagnosis, executor capability, and validation evidence align. HDSO provides an auditable skill lifecycle for frozen action agents rather than an unconstrained memory accumulation procedure.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。