研究自演化技能系统在多轮反馈中的真实效果，发现失败数据至关重要。
Rethinking Self-Evolving Agent Skills: Feedback Dynamics over Multiple Rounds
- 固定配置下对比成功、失败或混合反馈对技能演化的影响
- 仅55/388个候选技能通过验证，11/14设置中演化技能提升测试表现
- 失败轨迹对优化关键，但不同任务最优反馈策略不一致
自演化技能系统旨在通过执行反馈实现无模型修改的持续技能更新。然而，何时进一步演化有效、成功与失败轨迹如何共同影响修正过程，以及额外测试时计算能否达到相同收益仍不明确。为此，我们在五个基准和三个模型上构建了受控评估框架，共进行42次反馈实验，覆盖14个模型-基准组合。在每个组合中，固定执行器与优化器配置、修订流程、验证规则及轮次预算，仅改变反馈内容：仅成功、仅失败或混合反馈。演化过程稀疏：388个候选中仅55个达成字节不同的最佳验证结果。基于验证的选择在14个设置中有11个采纳了演化技能，其中9个提升了发布测试性能。所有11个成功选择均来自包含失败轨迹的反馈条件，但正常与仅失败反馈的相对表现因任务而异。验证与下游测试、鲁棒性及迁移评估结果常不一致。更广泛的SearchQA分析（覆盖8个模型）显示类似稀疏且依赖反馈的动态。GPT-5.5测试时扩展控制显示，理想并行采样仅落后0.43分，但在SpreadsheetBench仍落后30.96分；顺序精炼无法挽回损失。总体而言，持续技能演化应被理解为稀疏、验证过滤的搜索，其回报具有模型与基准依赖性，而非每轮递增的改进。
原文摘要 · Abstract (English)
Self-evolving skill systems promise to improve agents by turning execution feedback into persistent skill updates without changing the underlying model. Yet it remains unclear when further evolution helps, how successful and failed trajectories shape revision, and whether extra test-time computation can recover the same gains. To address these questions, we present a controlled evaluation framework across five benchmarks and three models. Our primary study contains 42 feedback runs across 14 supported model-benchmark settings. Within each setting, we hold the executor and optimizer configuration, revision procedure, validation rule, and round budget fixed, while varying only the feedback shown to the optimizer: successes and failures (Normal), failures only, or successes only. Evolution is sparse: only 55 of 388 candidates establish byte-distinct validation bests. Validation-based selection chooses an evolved skill in 11 of 14 settings, nine of which improve released-test performance. All 11 selections come from feedback conditions that include failed trajectories, although the relative ranking of Normal and Fail-only varies across settings. Validation and downstream evaluations on test, robustness, and transfer sometimes favor different feedback views. A broader SearchQA analysis covering eight models shows similarly sparse, feedback-dependent dynamics. In the GPT-5.5 test-time-scaling controls, oracle Parallel Sampling comes within 0.43 points of the evolved SearchQA skill but remains 30.96 points behind on SpreadsheetBench; Sequential Refinement recovers neither gain. Overall, persistent skill self-evolution is better understood as sparse, validation-filtered search with model- and benchmark-dependent returns, rather than steady improvement from additional rounds. The implementation is available at https://github.com/HKUST-KnowComp/rethinkskill.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。