让智能体的引导框架更通用高效，5次迭代提升代码任务通过率12个百分点。
HarnessCompass: Guiding Automatic Harness Evolution toward Generalizable and Effective Agent Harnesses
- 分步优化各组件，避免干扰，只允许泛化性修改。
- 用智能体自反馈补充轨迹信号，提升进化准确性。
- 可跨任务、跨模型迁移，适合需要强泛化的应用。
引导框架设计对大语言模型在可执行环境中的感知、推理与行动能力至关重要。现有自动引导演化方法常过度拟合演化任务，仅依赖轨迹信号，并联合优化各组件，导致组件间相互干扰。本文提出HarnessCompass，基于约束演化、主动反馈与分组件优化。首先施加全局约束，仅允许任务无关的泛化性修改；其次引入智能体自身关于引导使用的第一人称主动反馈，丰富演化信号；最后分步优化各组件后整合，减少干扰同时保留协同效应。在SWE-bench Verified上使用GPT-5.4，HarnessCompass仅用5次迭代将Pass@1从54%提升至66%，效果和效率均优于AHE。演化后的引导框架在未见任务和其他模型间有效迁移，显著优于现有方法的泛化能力。
原文摘要 · Abstract (English)
Harness design plays a critical role in agent performance by shaping how large language models (LLMs) perceive, reason over, and act within executable environments. Recent work has proposed automatic harness evolution, which iteratively improves the harness from agent--environment interactions. However, existing methods often overfit to the evolution tasks, rely exclusively on trajectory-derived signals, and optimize harness components jointly, causing interference across components. We propose HarnessCompass, a novel automatic harness evolution framework built around constrained evolution, proactive feedback, and component-wise optimization. HarnessCompass first enforces global constraints on evolution, restricting modifications to task-agnostic harness changes that generalize beyond the evolution tasks. It then augments trajectory-derived evidence with proactive first-person feedback from the agent about harness usage, yielding richer signals for evolution. Finally, it decouples the optimization of different harness components before consolidating them into a unified harness, reducing cross-component interference while preserving component synergy. On SWE-bench Verified with GPT-5.4, HarnessCompass improves Pass@1 from 54\% to 66\% in only 5 evolution iterations, outperforming AHE in both effectiveness and evolution efficiency. In addition, the evolved harness transfers effectively to held-out tasks and other models, demonstrating substantially stronger generalization than prior automatic harness evolution methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。