用生成测试代码的方法,精准识别代码改动是否真正无语义改变。
SemaDiff: Identifying Semantic-Changing Commits with Generated Code and Tests

- 通过大模型生成测试调用代码,统一测试前后版本的执行路径。
- 在183个提交中准确区分76%的语义不变与语义改变提交。
- 特别适合做缺陷定位、回滚分析和补丁移植的开发者使用。
区分语义不变提交与语义改变提交仍是软件仓库挖掘中的开放挑战。现有方法虽能准确检测重构提交,但无法确保提交仅含语义不变操作,而不夹杂行为改变。这一局限影响调试、故障定位、缺陷数据集构建、回滚分析及补丁回溯等任务。为此,我们提出 SemaDiff,一种基于行为分析的语义不变提交识别新方法:通过对比提交前后版本在相似测试下的执行结果。由于重构影响的代码往往难以测试且版本间差异大,SemaDiff 使用大语言模型生成额外的调用方法作为测试目标。给定一个提交,它分析 diff 以识别修改代码,并提取未改变的调用代码;随后生成一个附加依赖类来驱动两个版本中的被改代码,并自动生成其测试用例。如此可获得相同测试集用于跨版本行为比较。只有当所有生成测试在两版本中输出一致时,才判定为语义不变提交。为评估该方法,我们手动构建并标注了来自知名开源 Java 项目共183个提交的数据集。结果显示,SemaDiff 在约76%的案例中准确区分了语义不变与语义改变提交,且对语义改变提交的检测精度达100%。
原文摘要 · Abstract (English)
Distinguishing semantic-preserving commits from changing ones remains an open challenge in software repository mining. While existing approaches detect refactoring commits accurately, they cannot ensure that a commit is purely semantic-preserving, without any interleaving behaviour-changing modification. This limitation can impact several tasks, such as debugging, fault localisation, bug dataset construction, rollback analysis, and bug fixes backporting. To fill this gap, we propose SemaDiff, a novel approach for identifying semantic-preserving commits through behaviour-based analysis; comparison of similar test execution on pre- and post-commit versions. As code impacted by the refactoring is often hard to test and different accross both versions, we propose generating additional calling methods to that code, which serve as testing target. Given a commit, SemaDiff analyses the diff to identify modified code and extracts unchanged dependent code that calls it. It then generates an additional dependent class using a large language model to exercise the changed code in both versions, and automatically generates tests for the dependent code. This way, we obtain the same tests for the different code versions, enabling the behavioural-difference detection. The commit is classified as semantic-preserving only if all generated tests produce identical outcomes across the two versions. To evaluate SemaDiff, we construct and annotate manually a dataset of 183 commits, gathered from well-known open-source Java projects. The obtained results show that SemaDiff distinguishes accurately semantic-preserving from -- changing commits in about 76% of the cases, with a 100% precision in semantic-changing commit detection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。