arXiv:2608.02677cs.SEcs.AI2026-08综述

分离风险评估与决策策略,提升大模型代码审查准确性

When Policies Change Probabilities: Modular Decision-Making for LLM Code Review

  • 构建模块化流程:先独立评估风险,再结合监控得分和成本决策
  • 高误判代价下,原系统概率估计偏差达13.6至16.9个百分点
  • 新方法在等成本下接受率58%-68%,损失降低0.073/问题,适合实际部署

大模型代码审查常将风险评估与决策动作合并于单一提示中。我们通过15,792条对720个补丁的响应(每项仓库问题含一个通过、一个失败的测试用例)检验四种已部署接口是否保持风险与成本的分离。在固定补丁与监控证据的前提下,将等成本策略替换为10:1误接受代价策略,平均报告失败概率上升13.6至16.9个百分点。所有审查者在高代价提示下的行为均劣于全拒策略。将相同高代价规则应用于等成本下获取的概率,可降低四套系统的损失,表明概率提取本身导致额外损失。我们提出一种模块化管道:独立提取风险,融合外部监控得分,并在代码中应用成本。相比校准后的仅审查者评分,该管道提升了平均概率准确性,在等成本下每问题损失减少0.073,接受率58%~68%;在10:1代价下则拒绝全部,表现等同于全拒。这表明下游策略会改变其应使用的概率,需分别评估风险、外部证据与决策行动。

原文摘要 · Abstract (English)

LLM code reviewers often estimate patch risk and make approval decisions in one prompt. A probability should depend on evidence; costs should determine the action taken from it. We test whether four deployed reviewer interfaces preserve this separation using 15,792 responses on 720 candidate patches, with one that passed and one that failed an archived test harness for each of 360 repository issues. In matched calls with the patch and monitor evidence fixed, replacing an equal-cost policy with a 10:1 false-accept policy changes reported failure probabilities by 13.6 to 16.9 percentage points on average. For every reviewer, the actions returned under the high-cost prompt are worse than rejecting all patches. Applying the same high-cost rule to probabilities elicited under equal costs reduces loss for all four systems, showing that probability elicitation itself contributes to the excess loss. We also evaluate a modular pipeline that elicits risk without policy information, combines an independent monitor score, and applies costs in code. Relative to calibrated reviewer-only scores, the pipeline improves average probability accuracy and, at equal costs, reduces mean loss by .073 per issue while accepting 58 to 68% of patches. At 10:1, it accepts none and matches reject-all. Downstream policy can therefore change the probability it is meant to use, motivating separate evaluation of risk, outside evidence, and action.

代码审查大模型决策分离风险评估

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。