用大模型自动生成生产代码断言，提升可维护性与调试效率。
ASSERTIFY: Utilizing Large Language Models to Generate Assertions for Production Code
- 基于提示工程和少样本学习，利用大模型生成上下文相关的断言。
- 在2810个方法上测试，断言结构相似度平均达ROUGE-L 0.526。
- 适合需要自动化代码验证的开发者与大型项目维护团队。
生产断言是嵌入代码中的语句，帮助开发者验证对代码的假设，有助于调试、提供文档并提升代码理解。当前研究多聚焦于单元测试断言生成，依赖静态分析或深度学习，但难以满足生产断言的需求。本文提出Assertify，一个端到端自动化工具，利用大语言模型（LLMs）结合提示工程与少样本学习，生成符合开发习惯的生产断言。通过构建上下文丰富的提示，模拟开发者编写断言的思维过程。为评估效果，从GitHub上的22个成熟Java仓库中爬取2,810个方法构建数据集。实验表明，少样本学习生成的断言平均ROUGE-L得分达0.526，显示出与人工断言较高的结构相似性。该研究展示了大模型在生成类人化生产断言方面的潜力。
原文摘要 · Abstract (English)
Production assertions are statements embedded in the code to help developers validate their assumptions about the code. They assist developers in debugging, provide valuable documentation, and enhance code comprehension. Current research in this area primarily focuses on assertion generation for unit tests using techniques, such as static analysis and deep learning. While these techniques have shown promise, they fall short when it comes to generating production assertions, which serve a different purpose. This preprint addresses the gap by introducing Assertify, an automated end-to-end tool that leverages Large Language Models (LLMs) and prompt engineering with few-shot learning to generate production assertions. By creating context-rich prompts, the tool emulates the approach developers take when creating production assertions for their code. To evaluate our approach, we compiled a dataset of 2,810 methods by scraping 22 mature Java repositories from GitHub. Our experiments demonstrate the effectiveness of few-shot learning by producing assertions with an average ROUGE-L score of 0.526, indicating reasonably high structural similarity with the assertions written by developers. This research demonstrates the potential of LLMs in automating the generation of production assertions that resemble the original assertions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。