用大模型生成代码审查数据,让新语言也能自动判断是否需人工审查。
Code Review Without Borders: Evaluating Synthetic vs. Real Data for Review Recommendation
- 用大模型将主流语言代码转为新兴语言,生成合成训练数据。
- 合成数据训练的模型在低资源场景下表现接近真实数据训练结果。
- 适合快速迭代的技术栈,尤其缺乏标注数据的项目团队使用。
自动化判断代码变更是否需要人工审查对保障现代开发流程中的软件质量至关重要。然而,新编程语言和框架的出现带来关键瓶颈:尽管大量未标注代码易获取,但用于训练监督模型的标注数据严重不足。本文利用大语言模型(LLMs)将高资源语言的代码变更翻译为低资源或新兴语言中的等价变更,生成标注稀缺场景下的合成训练数据。我们假设,尽管LLMs已从可用的未标注代码中学习了新语言的语法与语义,但尚未完全掌握这些新兴生态中哪些代码变更具有重要性或值得审查。为此,我们使用LLMs生成合成变更样本,并基于这些数据训练监督分类器。我们在多个GitHub仓库和语言对上系统比较了该类模型与基于真实标注数据训练模型的性能。实验表明,由LLM生成的合成数据能有效启动审查推荐系统,在低资源环境下显著缩小性能差距。该方法为在缺乏标注数据的情况下扩展自动化代码审查能力提供了可扩展路径。
原文摘要 · Abstract (English)
Automating the decision of whether a code change requires manual review is vital for maintaining software quality in modern development workflows. However, the emergence of new programming languages and frameworks creates a critical bottleneck: while large volumes of unlabelled code are readily available, there is an insufficient amount of labelled data to train supervised models for review classification. We address this challenge by leveraging Large Language Models (LLMs) to translate code changes from well-resourced languages into equivalent changes in underrepresented or emerging languages, generating synthetic training data where labelled examples are scarce. We assume that although LLMs have learned the syntax and semantics of new languages from available unlabelled code, they have yet to fully grasp which code changes are considered significant or review-worthy within these emerging ecosystems. To overcome this, we use LLMs to generate synthetic change examples and train supervised classifiers on them. We systematically compare the performance of these classifiers against models trained on real labelled data. Our experiments across multiple GitHub repositories and language pairs demonstrate that LLM-generated synthetic data can effectively bootstrap review recommendation systems, narrowing the performance gap even in low-resource settings. This approach provides a scalable pathway to extend automated code review capabilities to rapidly evolving technology stacks, even in the absence of annotated data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。