用对比学习提升代码生成检测精度，准确率最高达94.65%
LLMSniffer: Detecting LLM-Generated Code via GraphCodeBERT and Supervised Contrastive Learning
- 基于GraphCodeBERT与两阶段对比学习，结合注释删除预处理
- 在两个数据集上准确率分别提升至78%和94.65%
- 适合关注代码安全与学术诚信的研究者使用
大型语言模型在软件开发中的广泛应用，使得区分人工智能生成代码与人工编写代码成为亟需解决的问题，涉及学术诚信、代码质量保障和软件安全。我们提出LLMSniffer检测框架，通过两阶段监督对比学习微调GraphCodeBERT，结合注释移除预处理和MLP分类器。在GPTSniffer与Whodunit两个基准数据集上评估显示，准确率分别从70%提升至78%（F1: 68%→78%），从91%提升至94.65%（F1: 91%→94.64%）。t-SNE可视化表明对比微调生成了分离良好且紧凑的嵌入表示。我们公开模型检查点、数据集、代码及在线交互演示，以推动后续研究。
原文摘要 · Abstract (English)
The rapid proliferation of Large Language Models (LLMs) in software development has made distinguishing AI-generated code from human-written code a critical challenge with implications for academic integrity, code quality assurance, and software security. We present LLMSniffer, a detection framework that fine-tunes GraphCodeBERT using a two-stage supervised contrastive learning pipeline augmented with comment removal preprocessing and an MLP classifier. Evaluated on two benchmark datasets - GPTSniffer and Whodunit - LLMSniffer achieves substantial improvements over prior baselines: accuracy increases from 70% to 78% on GPTSniffer (F1: 68% to 78%) and from 91% to 94.65% on Whodunit (F1: 91% to 94.64%). t-SNE visualizations confirm that contrastive fine-tuning yields well-separated, compact embeddings. We release our model checkpoints, datasets, codes and a live interactive demo to facilitate further research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。