不同大模型生成的代码有独特结构特征，可精准识别来源。
The Hidden DNA of LLM-Generated JavaScript: Structural Patterns Enable High-Accuracy Authorship Attribution
- 分析5万份大模型生成的JavaScript代码，发现模型特有结构模式。
- 新模型CodeT5-JSA在五类、十类、二十类识别任务中准确率达95.8%、94.6%、88.5%。
- 即使代码被修改或去注释，仍能有效识别来源，适合安全审计与责任追踪。
本文首次开展大规模研究，探索大语言模型（LLMs）生成的JavaScript代码是否能揭示其来源模型，实现可靠的作者归属与模型指纹识别。随着AI生成代码迅速普及，归属识别在漏洞检测、恶意内容标记和责任追溯中至关重要。我们发现，即使同家族或同参数规模的模型也留有独特风格痕迹。为此，构建了包含20个大模型的LLM-NodeJS数据集，每份代码生成四种变体，共25万条唯一样本，并提供JSIR和AST两种表示形式，支持多样化研究。基于该数据集，对比传统机器学习与微调的Transformer编码器，提出源自7.7亿参数CodeT5的定制架构CodeT5-JSA（移除解码器，修改分类头），在五类、十类、二十类归属任务中分别达到95.8%、94.6%、88.5%准确率，优于BERT、CodeBERT、Longformer等模型。结果表明，分类器捕捉的是程序数据流与结构中的深层风格规律，而非表层特征。即便经过代码混淆、注释删除和重度变换，归属依然有效。为促进开放科学，所有数据集、Colab训练脚本及材料已开源至GitHub：https://github.com/LLM-NodeJS-dataset。
原文摘要 · Abstract (English)
In this paper, we present the first large-scale study exploring whether JavaScript code generated by Large Language Models (LLMs) can reveal which model produced it, enabling reliable authorship attribution and model fingerprinting. With the rapid rise of AI-generated code, attribution is playing a critical role in detecting vulnerabilities, flagging malicious content, and ensuring accountability. While AI-vs-human detection usually treats AI as a single category we show that individual LLMs leave unique stylistic signatures, even among models belonging to the same family or parameter size. To this end, we introduce LLM-NodeJS, a dataset of 50,000 Node.js back-end programs from 20 large language models. Each has four transformed variants, yielding 250,000 unique JavaScript samples and two additional representations (JSIR and AST) for diverse research applications. Using this dataset, we benchmark traditional machine learning classifiers against fine-tuned Transformer encoders and introduce CodeT5-JSA, a custom architecture derived from the 770M-parameter CodeT5 model with its decoder removed and a modified classification head. It achieves 95.8% accuracy on five-class attribution, 94.6% on ten-class, and 88.5% on twenty-class tasks, surpassing other tested models such as BERT, CodeBERT, and Longformer. We demonstrate that classifiers capture deeper stylistic regularities in program dataflow and structure, rather than relying on surface-level features. As a result, attribution remains effective even after mangling, comment removal, and heavy code transformations. To support open science and reproducibility, we release the LLM-NodeJS dataset, Google Colab training scripts, and all related materials on GitHub: https://github.com/LLM-NodeJS-dataset.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。