用离散嵌套二元组检测大模型生成的Java代码，准确率超96%
Detection of LLM-Generated Java Code Using Discretized Nested Bigrams
- 提出离散嵌套二元组频次特征，将稀疏代码信息映射到密集桶中
- 在976个GPT生成的Java文件上达96%准确率，显著优于72%的API基线
- 适用于学术诚信与网络安全场景，可扩展至超万作者的大规模数据
大型语言模型（LLMs）被广泛用于生成代码，推动了代码生成检测工具的发展，以保障学术诚信和网络安全。本文将代码作者归属问题视为二分类任务，结合特征识别与提取。提出新的离散嵌套二元组频次特征，适用于不同大小的代码组。相比已有方法，该方法通过将稀疏信息表示为密集成员桶，提升了性能。实验表明，该方法在976个GPT-3.5和GPT-4生成的Java代码片段上，准确率超过96%，显著优于常用GPT代码检测API（72%）和基线特征（79%）。在40作者数据集上，亦优于三项先前工作。该方法可扩展至大规模数据，在76,089个文件、超1,000作者的GPT-4o数据上实现99%准确率与0.999 AUC，使用227个特征。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are currently used extensively to generate code by professionals and students, motivating the development of tools to detect LLM-generated code for applications such as academic integrity and cybersecurity. We address this authorship attribution problem as a binary classification task along with feature identification and extraction. We propose new Discretized Nested Bigram Frequency features on source code groups of various sizes. Compared to prior work, improvements are obtained by representing sparse information in dense membership bins. Experimental evaluation demonstrated that our approach significantly outperformed a commonly used GPT code-detection API and baseline features, with accuracy exceeding 96% compared to 72% and 79% respectively in detecting GPT-rewritten Java code fragments for 976 files with GPT 3.5 and GPT4 using 12 features. We also outperformed three prior works on code author identification in a 40-author dataset. Our approach scales well to larger data sets, and we achieved 99% accuracy and 0.999 AUC for 76,089 files and over 1,000 authors with GPT 4o using 227 features.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。