用结构化数据和可解释奖励训练大模型，提升代码审查的准确性和实用性。
Sphinx: Benchmarking and Modeling for LLM-Driven Pull Request Review
- 通过对比伪修改与已合并代码生成带上下文的评论数据
- 在检查清单覆盖度上比现有模型最高提升40%
- 适合想构建真实可用代码审查系统的研发团队
代码审查对保障软件质量至关重要，但自动化仍面临噪声监督、上下文理解不足和评估指标薄弱等挑战。我们提出Sphinx，一个面向大模型代码审查的统一框架，包含三项核心组件：(1) 基于伪修改与已合并代码对比的结构化数据生成管道，生成语义丰富、上下文相关的评审意见；(2) 基于检查清单的评估基准，从可操作验证点覆盖率评估评审质量，超越传统如BLEU等表面指标；(3) 检查清单奖励策略优化（CRPO），采用规则化、可解释的奖励机制，使模型行为契合实际评审实践。大量实验表明，使用Sphinx训练的模型在评审完整性和精确度上达到业界最优，检查清单覆盖率相比专有及开源基线最高提升40%。Sphinx使模型不仅流畅，更具备上下文感知、技术精准和实际部署能力。数据将在评审后公开。
原文摘要 · Abstract (English)
Pull request (PR) review is essential for ensuring software quality, yet automating this task remains challenging due to noisy supervision, limited contextual understanding, and inadequate evaluation metrics. We present Sphinx, a unified framework for LLM-based PR review that addresses these limitations through three key components: (1) a structured data generation pipeline that produces context-rich, semantically grounded review comments by comparing pseudo-modified and merged code; (2) a checklist-based evaluation benchmark that assesses review quality based on structured coverage of actionable verification points, moving beyond surface-level metrics like BLEU; and (3) Checklist Reward Policy Optimization (CRPO), a novel training paradigm that uses rule-based, interpretable rewards to align model behavior with real-world review practices. Extensive experiments show that models trained with Sphinx achieve state-of-the-art performance on review completeness and precision, outperforming both proprietary and open-source baselines by up to 40\% in checklist coverage. Together, Sphinx enables the development of PR review models that are not only fluent but also context-aware, technically precise, and practically deployable in real-world development workflows. The data will be released after review.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。