让大模型在无法保存状态的环境下也能高效试错，显著提升编程任务成功率。
Guided Search Strategies in Non-Serializable Environments with Applications to Software Engineering Agents
- 用学习到的估值函数指导搜索，分两步策略优化路径选择。
- 在SWE-bench上使Qwen-72B成功率翻倍至40.8%，达开源模型新高。
- 方法可迁移至闭源模型如GPT-4o，适合开发智能编程助手。
大型语言模型（LLMs）在复杂多步任务中表现优异，但在多次尝试中性能波动大。通过测试时引导搜索可缩小平均与最优表现差距，但传统方法（如MCTS）不适用于无法序列化的环境（如Docker容器）。本文研究两种适配此类环境的搜索策略：1步前瞻和轨迹选择，均基于学习的行动价值函数。在软件工程代理的重要基准SWE-bench Verified上，所提方法使微调后的Qwen-72B平均成功率提升一倍至40.8%，创开源模型新纪录。同时，该方法对更先进的闭源模型GPT-4o也有效，实现相似性能提升。
原文摘要 · Abstract (English)
Large language models (LLMs) have recently achieved remarkable results in complex multi-step tasks, such as mathematical reasoning and agentic software engineering. However, they often struggle to maintain consistent performance across multiple solution attempts. One effective approach to narrow the gap between average-case and best-case performance is guided test-time search, which explores multiple solution paths to identify the most promising one. Unfortunately, effective search techniques (e.g. MCTS) are often unsuitable for non-serializable RL environments, such as Docker containers, where intermediate environment states cannot be easily saved and restored. We investigate two complementary search strategies applicable to such environments: 1-step lookahead and trajectory selection, both guided by a learned action-value function estimator. On the SWE-bench Verified benchmark, a key testbed for agentic software engineering, we find these methods to double the average success rate of a fine-tuned Qwen-72B model, achieving 40.8%, the new state-of-the-art for open-weights models. Additionally, we show that these techniques are transferable to more advanced closed models, yielding similar improvements with GPT-4o.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。