arXiv:2603.03329cs.CLcs.AI2026-03被引 46

小模型自动生成代码防护罩,让智能体不犯错且超越大模型。

AutoHarness: improving LLM agents by automatically synthesizing a code harness

论文配图:AutoHarness: improving LLM agents by automatically synthesizing a code harness
图 1 · 摘自论文原文
  • 用少量迭代反馈,让小模型自动写出防止错误动作的代码防护层。
  • 在145个文本游戏中完全避免非法操作,性能超过更大模型。
  • 可生成完整决策代码,无需运行大模型,节省成本更高效。

尽管近年来语言模型取得显著进展,但作为智能体使用时,常会执行不符合当前状态甚至被外部环境严格禁止的动作。例如,在最近的Kaggle GameArena国际象棋比赛中,78%的Gemini-2.5-Flash失败源于非法走法。通常需人工编写“防护罩”代码来避免此类问题。本文证明,Gemini-2.5-Flash可通过少量迭代式代码优化,自动合成此类代码防护层,仅凭环境反馈即可实现。该防护层在145个TextArena游戏(含单人与双人)中彻底杜绝非法动作,使较小的Gemini-2.5-Flash模型表现超越更大的Gemini-2.5-Pro。进一步推进后,该方法能让模型生成完整的代码策略,从而在决策时完全无需调用语言模型。生成的代码策略在16个TextArena单人游戏中获得比Gemini-2.5-Pro和GPT-5.2-High更高的平均奖励。结果表明,用小模型自动生成定制化代码防护或完整策略,不仅能超越大模型,还更具成本效益。

原文摘要 · Abstract (English)

Despite significant strides in language models in the last few years, when used as agents, such models often try to perform actions that are not just suboptimal for a given state, but are strictly prohibited by the external environment. For example, in the recent Kaggle GameArena chess competition, 78% of Gemini-2.5-Flash losses were attributed to illegal moves. Often people manually write "harnesses" around LLMs to prevent such failures. In this paper, we demonstrate that Gemini-2.5-Flash can automatically synthesize such a code harness, using a small number of rounds of iterative code refinement given feedback from the (game) environment. The resulting harness prevents all illegal moves in 145 different TextArena games (both 1-player and 2-player), enabling the smaller Gemini-2.5-Flash model to outperform larger models, such as Gemini-2.5-Pro. Pushing our technique to the limit, we can get Gemini-2.5-Flash to generate the entire policy in code, thus eliminating the need to use the LLM at decision making time. The resulting code-policy receives a higher average reward than Gemini-2.5-Pro and GPT-5.2-High on 16 TextArena 1-player games. Our results show that using a smaller model to synthesize a custom code harness (or entire policy) can outperform a much larger model, while also being more cost effective.

智能体代码生成强化学习效率优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。