用小模型做代码规划点评，让大模型更高效地写代码。
Steer, Don't Solve: Training Small Critic Models for Large Code Agents
- 训练小型评论模型专门评估代码规划错误并给出指导
- 4B/8B小模型使大模型任务解决率提升16%以上
- 减少推理步数，降低单例成本至原价的43%
编码任务复杂，需涵盖从高层规划到低层实现的多种能力。尽管编码代理被优化以兼顾多项能力，但如高层规划等单项能力可能存在不同最优解，仍是主要瓶颈。为此，我们训练了一个专门用于高层规划的独立评论模型，在推理阶段引导编码代理。通过构建SFT和DPO数据集，使评论模型能够识别编码代理的错误，并提供清晰、正确的高层指导，而不生成具体动作。实验表明，经微调的4B和8B评论模型显著提升了6个更大编码代理的性能（例如在SWE-Bench Verified上使GLM-4.7-Flash-30B-A3B和GPT-OSS-120B的解决率分别提高16.0%和14.4%）。该评论模型还减少了部分编码代理的总推理成本，例如将GPT-OSS-20B的每例推理成本从$0.07降至$0.03。
原文摘要 · Abstract (English)
Coding tasks are typically complicated and require multiple capabilities, ranging from high-level planning to low-level implementation. While coding agents are optimized for the joint capabilities, individual capabilities such as high-level planning may have different optima and remain a major bottleneck. To address this challenge, we train a separate critic model that is specialized in high-level planning to steer the coding agent in inference. We construct SFT and DPO data to train the critic model to identify errors made by the coding agent and provide correct and clear high-level guidance without generating concrete actions. Experiments show that our fine-tuned 4B and 8B critic models significantly improve the performance of 6 larger coding agents (e.g., improving the resolved rates of GLM-4.7-Flash-30B-A3B and GPT-OSS-120B by 16.0% and 14.4% on SWE-Bench Verified). The critic model also reduces the total inference costs for some coding agents by solving tasks in fewer steps (e.g., reducing the per-example inference cost for GPT-OSS-20B from \$0.07 to \$0.03). Code: https://github.com/shubhamrgandhi/critic-training
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。