arXiv:2507.10583cs.SEcs.AI2025-07EMNLP被引 33

构建首个百万级代码生成检测数据集,提升检测模型泛化与抗干扰能力

$\texttt{Droid}$: A Resource Suite for AI-Generated Code Detection

  • 构建覆盖7语言、43模型的百万级代码数据集,含人机协作与对抗样本
  • 现有检测器在跨语言跨领域时性能大幅下降,易被表面改写绕过
  • 引入对抗样本和不确定性重采样,显著提升检测器鲁棒性与泛化能力

本文构建了迄今最全面的开源代码生成检测数据集 $ exttt{DroidCollection}$,包含超百万代码样本、七种编程语言、43个编码模型输出,以及超过三个真实开发场景。数据集不仅包含纯人工智能生成代码,还涵盖人机协作代码和专为逃避检测设计的对抗样本。基于此,我们开发了 $ exttt{DroidDetect}$,一套使用多任务目标训练的编码器型检测器。实验表明,现有检测器在非训练域和跨语言场景下性能严重退化;多数检测器可被简单提示与对齐策略轻易绕过。但通过在少量对抗样本上训练,可有效缓解该问题。此外,我们验证了度量学习与基于不确定性的重采样方法在处理可能含噪分布时的有效性。

原文摘要 · Abstract (English)

In this work, we compile $\textbf{$\texttt{DroidCollection}$}$, the most extensive open data suite for training and evaluating machine-generated code detectors, comprising over a million code samples, seven programming languages, outputs from 43 coding models, and over three real-world coding domains. Alongside fully AI-generated samples, our collection includes human-AI co-authored code, as well as adversarial samples explicitly crafted to evade detection. Subsequently, we develop $\textbf{$\texttt{DroidDetect}$}$, a suite of encoder-only detectors trained using a multi-task objective over $\texttt{DroidCollection}$. Our experiments show that existing detectors' performance fails to generalise to diverse coding domains and programming languages outside of their narrow training data. Additionally, we demonstrate that while most detectors are easily compromised by humanising the output distributions using superficial prompting and alignment approaches, this problem can be easily amended by training on a small amount of adversarial data. Finally, we demonstrate the effectiveness of metric learning and uncertainty-based resampling as means to enhance detector training on possibly noisy distributions.

代码生成检测模型对抗样本多语言

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。