代码评估指标能有效检测不同程度的抄袭，部分表现甚至优于专用工具。
Can Code Evaluation Metrics Detect Code Plagiarism?
- 对比五种代码评估指标与两大抄袭检测工具，全面测试其检测能力。
- 无预处理时，CrystalBLEU等指标在低复杂度抄袭中表现优于JPlag；有预处理后更胜过Dolos。
- 在高难度抄袭（如L6）中，CrystalBLEU仍保持竞争力，适合教学场景使用。
代码抄袭检测（SCPD）在软件工程教育中对维护公平性与学术诚信至关重要。代码评估指标（CEMs）用于评估代码生成任务，但其能否可靠检测不同修改程度（L1-L6）的抄袭仍不明确。本文基于两个开源标注数据集ConPlag（原始版与模板无关版）和IRPlag，对五种CEMs——CodeBLEU、CrystalBLEU、RUBY、Tree Structured Edit Distance（TSED）、CodeBERTScore——进行对比实证研究。采用无需阈值的排序评价方法，评估整体、各数据集及各抄袭级别下的性能，并与SOTA抄袭检测工具JPlag和Dolos比较。结果表明：未预处理时，Dolos整体排名最高；而单个指标中，CrystalBLEU、CodeBLEU、RUBY优于JPlag。性能在L1最强，从L4开始下降，但CrystalBLEU在L6仍具竞争力。经预处理后，CrystalBLEU超越Dolos。在数据集层面，Dolos在ConPlag原始集表现最佳，其余数据集以CrystalBLEU为优。在抄袭级别上，Dolos在L4最强，而CrystalBLEU主导其余较难级别。结果表明，某些CEMs在排序指标上已可媲美专用工具。
原文摘要 · Abstract (English)
Source Code Plagiarism Detection (SCPD) plays an important role in maintaining fairness and academic integrity in software engineering education. Code Evaluation Metrics (CEMs) are developed for assessing code generation tasks. However, it remains unclear whether such metrics can reliably detect plagiarism across different levels of modification (L1-L6), increasing in complexity. In this paper, we perform a comparative empirical study using two open-source labelled datasets, ConPlag (raw and template-free versions) and IRPlag. We evaluate five CEMs, namely CodeBLEU, CrystalBLEU, RUBY, Tree Structured Edit Distance (TSED), and CodeBERTScore. The performance is evaluated using threshold-free ranking-based measures to assess overall, per dataset, and per-level plagiarism performance. The results are compared against state-of-the-art (SOTA) Source Code Plagiarism Detection Tools (SCPDTs), JPlag and Dolos. Our findings show that without preprocessing, Dolos achieves the highest overall ranking performance, while among the individual metrics, CrystalBLEU, CodeBLEU, and RUBY outperform JPlag. Performance is strongest at L1 and drops from L4 onward, while CrystalBLEU remains competitive on L6. With preprocessing, CrystalBLEU surpasses Dolos overall. Per dataset, Dolos achieved the best ranking on the ConPlag raw dataset, while CrystalBLEU was the best-performing metric on the remaining datasets. At the plagiarism levels, Dolos remains strongest on L4, while Crystal-BLEU leads most of the remaining difficult levels. These results indicate that CEMs are comparable to dedicated tools in terms of ranking metrics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。