让搜索模型专注决策,把记忆交给环境,性能提升11.4点。
Harness-1: Reinforcement Learning for Search Agents with State-Externalizing Harnesses

- 用外部记忆框架分离状态管理,让策略只做语义决策
- 在8个检索任务上平均准确率0.730,领先次强模型11.4点
- 跨领域迁移表现突出,适合需要泛化能力的搜索系统
搜索代理常被训练为对不断增长的对话记录进行策略决策:模型需决定如何搜索,同时记住已见内容、有用证据、未解决约束和已验证声明。我们指出,这种设定将大量可恢复的状态管理任务置于策略内部,迫使强化学习同时优化语义搜索与可由环境维护的记账工作。为此提出Harness-1,一个200亿参数的搜索代理(检索子代理),在具状态的搜索框架内通过强化学习训练。该框架在环境侧维护工作内存,包括候选池、重要性标记的精选集、紧凑证据链接、验证记录、压缩去重的观测数据及预算感知的上下文渲染。策略仅保留语义决策权:决定搜什么、留或删哪些文档、验证什么内容、何时停止。在涵盖网络、金融、专利和多跳问答的8个检索基准测试中,Harness-1实现0.730平均精选召回率,优于次强开源子代理11.4个百分点,且与更大规模前沿模型相当。其优势在未见的迁移基准上尤为显著,表明对显式搜索状态的强化学习能生成超越训练域的泛化检索行为。代码已开源。
原文摘要 · Abstract (English)
Search agents are often trained as policies over growing transcripts: the model must decide how to search while also remembering what it has seen, which evidence is useful, which constraints remain open, and which claims have actually been checked. We argue that this formulation puts too much routine state management inside the policy: reinforcement learning is forced to optimize both semantic search decisions and recoverable bookkeeping that the environment can maintain more reliably. We introduce Harness-1, a 20B search agent (retrieval subagent) trained with reinforcement learning inside a stateful search harness. The harness maintains environment-side working memory, including a candidate pool, an importance-tagged curated set, compact evidence links, verification records, compressed and deduplicated observations, and budget-aware context rendering. The policy retains the semantic decisions: what to search, which documents to keep or discard, what to verify, and when to stop. Across eight retrieval benchmarks spanning web, finance, patents, and multi-hop QA, Harness-1 achieves 0.730 average curated recall, outperforming the next strongest open search subagent by +11.4 points and remaining competitive with much larger frontier-model searchers. Its gains are especially strong on held-out transfer benchmarks, suggesting that reinforcement learning over explicit search state can produce retrieval behaviors that generalize beyond the training domains. Our code is available at https://github.com/pat-jj/harness-1.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。