针对代码审查自动化中缺陷定位不准的问题,提出一套更贴近真实开发流程的解决方案。
Towards Practical Defect-Focused Automated Code Review
- 用代码切片提取上下文,结合多角色大模型提升缺陷发现能力
- 在真实合并请求上实现缺陷检测率提升2倍,误报率降低10倍
- 设计适配工程师工作流的提示词,适合工业级C++代码审查场景
代码审查的复杂性推动了自动化评论生成的研究,但现有方法将任务简化为片段级代码到文本生成,并依赖BLEU等文本相似度指标评估,忽视了仓库上下文、真实合并请求评估和缺陷检测,限制了实用性。本文基于一家拥有近4亿日活用户的在线推荐服务公司的工业级C++代码库(含数十万行代码),分析了四个关键挑战:1)捕捉相关上下文,2)提升关键缺陷包含率(KBI),3)降低误报率(FAR),4)集成人工工作流。为此提出:1)基于代码切片的上下文提取算法,2)多角色大模型框架以增强KBI,3)过滤机制降低FAR,4)新型提示词设计优化人机交互。在历史故障报告中的真实合并请求上验证,该方法相较标准LLM提升2倍,较先前基线提升10倍。尽管结果聚焦于C++,但框架设计采用语言无关原则(如基于AST分析),具备广泛适用潜力。
原文摘要 · Abstract (English)
The complexity of code reviews has driven efforts to automate review comments, but prior approaches oversimplify this task by treating it as snippet-level code-to-text generation and relying on text similarity metrics like BLEU for evaluation. These methods overlook repository context, real-world merge request evaluation, and defect detection, limiting their practicality. To address these issues, we explore the full automation pipeline within the online recommendation service of a company with nearly 400 million daily active users, analyzing industry-grade C++ codebases comprising hundreds of thousands of lines of code. We identify four key challenges: 1) capturing relevant context, 2) improving key bug inclusion (KBI), 3) reducing false alarm rates (FAR), and 4) integrating human workflows. To tackle these, we propose 1) code slicing algorithms for context extraction, 2) a multi-role LLM framework for KBI, 3) a filtering mechanism for FAR reduction, and 4) a novel prompt design for better human interaction. Our approach, validated on real-world merge requests from historical fault reports, achieves a 2x improvement over standard LLMs and a 10x gain over previous baselines. While the presented results focus on C++, the underlying framework design leverages language-agnostic principles (e.g., AST-based analysis), suggesting potential for broader applicability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。