用多版本代码对比提升生成式AI代码可信度
N-Version Assessment and Enhancement of Generative AI
- 通过生成多个代码版本进行差异比对，替代单一版本评估
- 在大规模测试中验证，显著降低错误代码漏检率
- 适合关注AI代码质量的开发者与模型研究者
生成式AI（GAI）虽有望提升软件工程效率，但其在代码生成中的不可靠输出带来严峻挑战。若需大量验证与确认，可能抵消效率优势。本文提出一种“差异式GAI”（D-GAI）方法，利用GAI生成多个代码和测试版本，通过跨版本对比实现更可靠的评估。为此构建了大型软件观测平台LASSO，支持大规模代码与测试版本的执行与分析。LASSO可实现对GAI生成成果的严格评估，并适用于软件开发与GAI研究双重场景。
原文摘要 · Abstract (English)
Generative AI (GAI) holds great potential to improve software engineering productivity, but its untrustworthy outputs, particularly in code synthesis, pose significant challenges. The need for extensive verification and validation (V&V) of GAI-generated artifacts may undermine the potential productivity gains. This paper proposes a way of mitigating these risks by exploiting GAI's ability to generate multiple versions of code and tests to facilitate comparative analysis across versions. Rather than relying on the quality of a single test or code module, this "differential GAI" (D-GAI) approach promotes more reliable quality evaluation through version diversity. We introduce the Large-Scale Software Observatorium (LASSO), a platform that supports D-GAI by executing and analyzing large sets of code versions and tests. We discuss how LASSO enables rigorous evaluation of GAI-generated artifacts and propose its application in both software development and GAI research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。