用稀疏真实交互数据训练智能评判模型，提升代码生成效率与质量。
A Rubric-Supervised Critic from Sparse Real-World Outcomes
- 基于人类-代理交互轨迹提取24个行为特征，构建评分框架
- 在SWE-bench上使最佳重排效果提升15.9%，减少83%尝试次数
- 适合需要高效调试和自动化评估的代码生成研究者
学术代码代理评估多以可验证奖励（如单元测试通过）为标准，而真实场景中成功信号往往噪声大、延迟高且稀疏。本文提出一种从稀疏噪声交互数据中学习‘评判模型’的流程，该模型可用于强化学习训练或推理时的奖励估计。我们引入Critic Rubrics——一个包含24个可从交互轨迹中提取的行为特征的评分框架，结合半监督目标，联合预测这些特征与稀疏的人类反馈（若有）。实验表明，尽管主要依赖轨迹可观测特征和稀疏结果代理，该评判模型在SWE-bench上使Best@8相比Random@8提升15.9%，实现提前终止（减少83%尝试次数），并支持通过评判模型筛选训练数据轨迹。
原文摘要 · Abstract (English)
Academic benchmarks for coding agents tend to reward autonomous task completion, measured by verifiable rewards such as unit-test success. In contrast, real-world coding agents operate with humans in the loop, where success signals are typically noisy, delayed, and sparse. How can we bridge this gap? In this paper, we propose a process to learn a "critic" model from sparse and noisy interaction data, which can then be used both as a reward model for either RL-based training or inference-time scaling. Specifically, we introduce Critic Rubrics, a rubric-based supervision framework with 24 behavioral features that can be derived from human-agent interaction traces alone. Using a semi-supervised objective, we can then jointly predict these rubrics and sparse human feedback (when present). In experiments, we demonstrate that, despite being trained primarily from trace-observable rubrics and sparse real-world outcome proxies, these critics improve best-of-N reranking on SWE-bench (Best@8 +15.9 over Random@8 over the rerankable subset of trajectories), enable early stopping (+17.7 with 83% fewer attempts), and support training-time data curation via critic-selected trajectories.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。