让大模型的执行框架自动进化，提升任务表现。
Hierarchical Self-Improvement: A Framework for Task-Specific Evolvable Agent Harnesses
- 用分层自改进框架，让任务专用执行框架持续优化。
- 在多个任务上进步显著，最高提升39.3个百分点。
- 适合想提升冻结大模型任务能力的研究者。
现代大模型代理通常通过手动调整提示、工具或流程来改进，而模型的可执行结构（即‘框架’）在部署后常被视为固定不变。本文提出一种新范式：任务专属且可持续进化的框架。每个任务族维护专属框架，通过固定的任务注入接口热切换，并基于环境反馈重写。引入分层自改进（HSI）框架，单一冻结模型在三个层级运作：执行任务的框架、重写框架的演化器，以及重写演化策略的元演化器。通过思考开关设计，隔离框架进化对性能的影响。系统受两大限制：反馈保真度上限（需有效奖励信号）与基础模型能力上限（无法弥补模型缺陷）。在以 DeepSeek-V4-Flash-Preview 为骨干的 BALROG 上，HSI 在中等难度任务上持续提升：BabyAI +39.3%，Crafter +33.0%，TextWorld +25.0%，MiniHack +15.0%（原始进度百分比）。在未见过的 BabaIsAI 子集上表现优异：BreakStop 最佳测试达 0.98，GoTo 达 1.00（20% 未见数据）。超出模型能力范围的任务（NLE），框架进化无效。结果表明，任务专属框架进化是提升冻结大模型代理的有效路径，且具有明确的实证边界。代码已开源。
原文摘要 · Abstract (English)
Modern LLM agents are often improved by modifying prompts, tools, or workflows manually, while the executable scaffold surrounding the model---the \emph{harness}---is typically treated as a fixed artifact after deployment. This work studies an alternative where the harness is \emph{task-specific and continuously evolvable}: each task family maintains its own harness, which is hot-swapped across iterations through a fixed task-injection seam and rewritten using environment feedback. We introduce \textbf{Hierarchical Self-Improvement (HSI)}, a framework in which a single frozen LLM $M$ operates across three hierarchical scopes: a task harness $H$ that executes tasks, an evolver that rewrites $H$, and a meta-evolver that rewrites the evolver's strategy code under a frozen outer anchor. A thinking-on/off design isolates the contribution of harness evolution by disabling reasoning during task execution while enabling it during self-modification. HSI is bounded by two factors: a \emph{feedback-fidelity bound}, since evolution requires informative reward signals to guide selection, and a \emph{backbone capability bound}, since harness redesign cannot overcome limitations of the frozen model. On BALROG with DeepSeek-V4-Flash-Preview as the frozen backbone, HSI achieves consistent gains over the initial harness on moderate-difficulty tasks ($+39.3$ on BabyAI, $+33.0$ on Crafter, $+25.0$ on TextWorld, and $+15.0$ on MiniHack, all in raw \% Progress), while obtaining strong held-out generalization on BabaIsAI sub-suites ($0.98$ best-test on BreakStop and $1.00$ on GoTo from a $20\%$ unseen split). On tasks beyond the backbone's capability (NLE), harness evolution provides no improvement. These results demonstrate task-specific harness evolution as a viable axis for improving frozen LLM agents under clear empirical limits. Code is available at https://github.com/TailinZhou/hsi.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。