让大模型自主学习如何管理外部工作空间，提升长任务执行成功率。
EvoHarness-RL: Learning Self-Evolving Runtime Harness for Long-Horizon LLM Agents
- 用信念、进展、经验三要素构建可学习的外部状态，支持动态管理
- 在ALFWorld上实现96.9%任务成功率，显著优于传统方法
- 适合研究长时序推理、智能体系统设计的开发者与研究人员
长时序大模型智能体依赖外部执行环境来维护状态、追踪进度、调用工具、验证结果并复用经验。然而，有效使用外部工作空间面临两大挑战：从嘈杂交互轨迹中形成可靠状态，以及对状态访问进行运行时控制。现有方法多通过提示词或启发式规则处理，导致工作空间及其使用策略需人工设计。为此，本文提出EvoHarness-RL，将信念、进展、经验（BPE）作为策略可见的状态，通过监督微调教会基础模型构建有用外部状态，再利用成本感知的GRPO算法探索协调策略，在长时序交互中选择性读取、更新和整合状态。在ALFWorld任务环境中，基于Qwen3-8B模型的EvoHarness-RL达到96.9%的成功率，揭示出两个关键机制：‘状态退火’使模型内化重复使用的状态模式，减少对外部调用频率；‘状态演化’通过进度更新与经验融合，将状态优化为紧凑且任务自适应的基底结构。结果表明，长时序智能体需要可训练的状态构建与协调策略，而非仅依赖更强工具或更大记忆。
原文摘要 · Abstract (English)
Long-horizon LLM agents increasingly rely on external execution support to maintain state, track progress, invoke tools, verify outcomes, and reuse experience across interactions. However, effective harness use raises two coupled challenges: state formation from noisy interaction traces and runtime control over external-state access. Existing agents usually handle both through prompts, heuristics, or domain-specific conventions, leaving the external workspace and its usage policy manually engineered. To address this, we study the problem of harness policy learning, where agents learn harness policies offline and deploy them to construct and update external harness state online during runtime task execution. We introduce EvoHarness-RL, which exposes Belief, Progress, and Experience (BPE) as policy-facing harness state. Supervised harness fine-tuning teaches the base agent the harness action space and how to construct useful external state, while cost-aware GRPO explores coordination policies to selectively read, update, and consolidate that state during long-horizon interaction. Instantiated on ALFWorld with a Qwen3-8B LLM, EvoHarness-RL reaches 96.9% success and reveals two key dynamics: harness annealing, where training internalizes recurring harness-use patterns into the model policy and shifts the agent from frequent harness calls toward selective external-state access, and harness evolution, where progress updates and experience consolidation refine the harness into a compact, task-adaptive state substrate. These results suggest that long-horizon agents benefit from trainable policies for constructing and coordinating with external harness workspaces, beyond simply adding stronger tools or larger memories.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。