用统计检验方法零样本识别代码是否由大模型生成
Zero-Shot Attribution for Large Language Models: A Distribution Testing Approach
- 将代码归属问题转为分布检验，利用样本和密度估计进行判断
- 仅需约2000个样本，对多个大模型的区分准确率超0.9
- 适合需要验证代码来源可信性的开发者与安全研究人员
越来越多的代码来自大语言模型（LLMs）。本文研究如何通过假设检验对模型生成的代码进行溯源，利用成熟统计方法与理论保证。给定一组代码样本 $S$ 和一个可疑模型 $/mathcal{L}^*$，目标是评估 $S$ 来源于 $/mathcal{L}^*$ 的可能性。由于维度灾难，仅凭模型输出样本难以实现；为此，我们同时使用样本和模型的密度估计，这种访问方式在实践中常见。我们提出 $ extsf{Anubis}$，一种零样本归属工具，将归属问题建模为分布测试。在代码样本基准上的实验表明，$ extsf{Anubis}$ 在区分 DeepSeek-Coder、CodeGemma 与 Stable-Code 等模型时，仅需约 2000 个样本即可达到 ≥0.9 的 AUROC 分数。
原文摘要 · Abstract (English)
A growing fraction of all code is sampled from Large Language Models (LLMs). We investigate the problem of attributing code generated by language models using hypothesis testing to leverage established techniques and guarantees. Given a set of samples $S$ and a suspect model $\mathcal{L}^*$, our goal is to assess the likelihood of $S$ originating from $\mathcal{L}^*$. Due to the curse of dimensionality, this is intractable when only samples from the LLM are given: to circumvent this, we use both samples and density estimates from the LLM, a form of access commonly available. We introduce $\mathsf{Anubis}$, a zero-shot attribution tool that frames attribution as a distribution testing problem. Our experiments on a benchmark of code samples show that $\mathsf{Anubis}$ achieves high AUROC scores ( $\ge0.9$) when distinguishing between LLMs like DeepSeek-Coder, CodeGemma, and Stable-Code using only $\approx 2000$ samples.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。