arXiv:2602.02079cs.LGcs.SE2026-02Conference of the …被引 16

构建首个覆盖200万样本的代码生成检测基准,推动更鲁棒的AI代码识别技术。

AICD Bench: A Challenging Benchmark for AI-Generated Code Detection

  • 构建涵盖77个模型、11种语言的超大规模检测数据集
  • 在分布外场景下,现有方法检测准确率不足60%且对混合代码失效
  • 适合研究者评估模型泛化能力,或开发安全审计工具

大型语言模型(LLMs)日益具备生成功能性源代码的能力,引发作者归属、责任认定与安全风险等问题。尽管检测AI生成代码至关重要,但现有数据集和基准范围有限,通常仅限于同分布下的二分类任务。为填补这一空白,我们提出《AICD Bench》,目前最全面的AI生成代码检测基准。该基准包含200万条样本,覆盖77个模型、11个模型家族及9种编程语言,包括最新推理类模型。除规模外,AICD Bench引入三项真实检测任务:(i)在语言和领域分布偏移下进行鲁棒二分类;(ii)模型家族溯源,按架构谱系分组;(iii)细粒度的人机分类,涵盖人类、机器、混合及对抗性代码。对神经网络与传统检测器的广泛评估表明,当前性能远未达到实际可用水平,尤其在分布外场景以及处理混合或对抗性代码时表现显著下降。我们开放AICD Bench作为统一、具有挑战性的评估套件,以推动下一代鲁棒检测方法的发展。数据与代码已发布于 https://huggingface.co/AICD-bench。

原文摘要 · Abstract (English)

Large language models (LLMs) are increasingly capable of generating functional source code, raising concerns about authorship, accountability, and security. While detecting AI-generated code is critical, existing datasets and benchmarks are narrow, typically limited to binary human-machine classification under in-distribution settings. To bridge this gap, we introduce $\emph{AICD Bench}$, the most comprehensive benchmark for AI-generated code detection. It spans $\emph{2M examples}$, $\emph{77 models}$ across $\emph{11 families}$, and $\emph{9 programming languages}$, including recent reasoning models. Beyond scale, AICD Bench introduces three realistic detection tasks: ($\emph{i}$)~$\emph{Robust Binary Classification}$ under distribution shifts in language and domain, ($\emph{ii}$)~$\emph{Model Family Attribution}$, grouping generators by architectural lineage, and ($\emph{iii}$)~$\emph{Fine-Grained Human-Machine Classification}$ across human, machine, hybrid, and adversarial code. Extensive evaluation on neural and classical detectors shows that performance remains far below practical usability, particularly under distribution shift and for hybrid or adversarial code. We release AICD Bench as a $\emph{unified, challenging evaluation suite}$ to drive the next generation of robust approaches for AI-generated code detection. The data and the code are available at https://huggingface.co/AICD-bench}.

代码检测大模型安全基准测试

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。