arXiv:2412.14739cs.SEcs.LG2024-12中稿 · the 40th IEEE Inte…被引 13

对比五种模型在多数据集上的克隆检测表现,发现跨语言模型更稳健。

On the Use of Deep Learning Models for Semantic Clone Detection

  • 用多数据集+变异操作评估五种克隆检测模型的鲁棒性。
  • 跨语言模型C4在语义克隆数据集上表现领先,F1提升约7%。
  • 单语言模型对代码变异敏感,性能波动大,适合特定场景使用。

检测和追踪代码克隆可简化软件开发与维护任务,当代码片段变更时需同步更新所有副本。已有多种基于深度学习的克隆检测模型用于识别语法和语义克隆,主要在BigCloneBench数据集上评估。然而,类别不平衡及语义克隆数量少使BigCloneBench难以准确反映模型性能。研究人员也使用GoogleCodeJam、OJClone和SemanticCloneBench等数据集评估模型泛化能力。为克服现有数据集局限,新发布的GPTCloneBench结合GPT生成语义与跨语言克隆。本文提出多步骤评估方法,涵盖五种前沿克隆检测模型,利用现有基准数据集(包括GPTCloneBench)并引入变异操作分析模型能力。重点考察三种高性能单语言模型(ASTNN、GMN、CodeBERT)在BigCloneBench、SemanticCloneBench和GPTCloneBench的表现,并与跨语言模型(C4、CLCDSA)对比。结果显示,单语言模型在BigCloneBench上F1值高,但在SemanticCloneBench上表现差异达20%;而跨语言模型C4在SemanticCloneBench上表现显著优于其他模型(约7%),且在BigCloneBench和GPTCloneBench上性能相当。在变异数据集上,C4性能波动小于1%,远优于单语言模型的高变异性。

原文摘要 · Abstract (English)

Detecting and tracking code clones can ease various software development and maintenance tasks when changes in a code fragment should be propagated over all its copies. Several deep learning-based clone detection models have appeared in the literature for detecting syntactic and semantic clones, widely evaluated with the BigCloneBench dataset. However, class imbalance and the small number of semantic clones make BigCloneBench less ideal for interpreting model performance. Researchers also use other datasets such as GoogleCodeJam, OJClone, and SemanticCloneBench to understand model generalizability. To overcome the limitations of existing datasets, the GPT-assisted semantic and cross-language clone dataset GPTCloneBench has been released. However, how these models compare across datasets remains unclear. In this paper, we propose a multi-step evaluation approach for five state-of-the-art clone detection models leveraging existing benchmark datasets, including GPTCloneBench, and using mutation operators to study model ability. Specifically, we examine three highly-performing single-language models (ASTNN, GMN, CodeBERT) on BigCloneBench, SemanticCloneBench, and GPTCloneBench, testing their robustness with mutation operations. Additionally, we compare them against cross-language models (C4, CLCDSA) known for detecting semantic clones. While single-language models show high F1 scores for BigCloneBench, their performance on SemanticCloneBench varies (up to 20%). Interestingly, the cross-language model (C4) shows superior performance (around 7%) on SemanticCloneBench over other models and performs similarly on BigCloneBench and GPTCloneBench. On mutation-based datasets, C4 has more robust performance (less than 1% difference) compared to single-language models, which show high variability.

代码克隆深度学习模型评估跨语言

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。