跨语言跨生成器检测代码是否由AI生成
CoDet-M4: Detecting Machine-Generated Code in Multi-Lingual, Multi-Generator and Multi-Domain Settings
- 构建多语言多生成器多领域检测框架
- 在未见模型/语言/领域上仍保持高准确率
- 适合代码审查、学术诚信与安全评估场景
大型语言模型(LLMs)已显著提升代码生成效率,但对编程能力、伦理及评估公正性带来挑战,因此识别LLM生成代码至关重要。现有研究覆盖范围有限,仅涉及少数编程语言和生成器。为此,我们提出一个跨多语言、多生成器、多领域的代码生成检测框架。基于知名平台和LLM生成器的海量数据,结合严格的数据质量检查、特征工程,并对比传统机器学习模型、预训练语言模型(PLMs)及大模型在代码检测中的表现。我们在跨域场景下进行评估,包括作者身份识别、混合作者代码检测,以及对未见模型、领域和编程语言的泛化能力。大量实验表明,该框架能有效区分人类与LLM编写的代码,为该任务设立了新基准。
原文摘要 · Abstract (English)
Large language models (LLMs) have revolutionized code generation, automating programming with remarkable efficiency. However, these advancements challenge programming skills, ethics, and assessment integrity, making the detection of LLM-generated code essential for maintaining accountability and standards. While, there has been some research on this problem, it generally lacks domain coverage and robustness, and only covers a small number of programming languages. To this end, we propose a framework capable of distinguishing between human- and LLM-written code across multiple programming languages, code generators, and domains. We use a large-scale dataset from renowned platforms and LLM-based code generators, alongside applying rigorous data quality checks, feature engineering, and comparative analysis using evaluation of traditional machine learning models, pre-trained language models (PLMs), and LLMs for code detection. We perform an evaluation on out-of-domain scenarios, such as detecting the authorship and hybrid authorship of generated code and generalizing to unseen models, domains, and programming languages. Moreover, our extensive experiments show that our framework effectively distinguishes human- from LLM-written code and sets a new benchmark for this task.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。