构建多语言多场景的代码生成检测数据集，助力学术诚信与招聘公平
MultiAIGCD: A Comprehensive dataset for AI Generated Code Detection Covering Multiple Languages, Models,Prompts, and Scenarios
- 基于CodeNet生成三语言代码，覆盖六模型三提示词
- 含12万余条生成代码与3万余条人工代码，涵盖三大使用场景
- 支持跨模型跨语言检测评估，适合代码安全与教育研究者
随着大语言模型快速发展，其在代码生成中的应用日益广泛。这虽提升了开发效率，但也引发教育和招聘领域的信任危机。为此，本文提出MultiAIGCD数据集，覆盖Python、Java和Go三种语言。基于CodeNet的问题定义与人工代码，利用六种LLM和三种提示词生成代码样本，涵盖三大典型场景：(i) 根据问题描述生成代码，(ii) 修复人工代码的运行时错误，(iii) 修正错误输出。数据集共包含121,271条AI生成代码与32,148条人工编写代码。我们还对三种先进检测模型进行基准测试，评估其在跨模型、跨语言等场景下的表现。相关数据与代码已公开，以推动该领域研究。
原文摘要 · Abstract (English)
As large language models (LLMs) rapidly advance, their role in code generation has expanded significantly. While this offers streamlined development, it also creates concerns in areas like education and job interviews. Consequently, developing robust systems to detect AI-generated code is imperative to maintain academic integrity and ensure fairness in hiring processes. In this study, we introduce MultiAIGCD, a dataset for AI-generated code detection for Python, Java, and Go. From the CodeNet dataset's problem definitions and human-authored codes, we generate several code samples in Java, Python, and Go with six different LLMs and three different prompts. This generation process covered three key usage scenarios: (i) generating code from problem descriptions, (ii) fixing runtime errors in human-written code, and (iii) correcting incorrect outputs. Overall, MultiAIGCD consists of 121,271 AI-generated and 32,148 human-written code snippets. We also benchmark three state-of-the-art AI-generated code detection models and assess their performance in various test scenarios such as cross-model and cross-language. We share our dataset and codes to support research in this field.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。