arXiv:2503.01659cs.CL2025-03被引 10

通过风格指纹识别大模型生成文本,精准率达99.88%。

Detecting Stylistic Fingerprints of Large Language Models

  • 构建三分类器集成模型,从文本风格识别生成模型
  • 在四类大模型上实现0.9988精确率与0.0004误报率
  • 可区分已见与未见模型,适合版权保护与内容溯源

大语言模型(LLMs)具有独特且一致的风格指纹,即使被要求以不同文风写作也如此。检测这些指纹对保护知识产权、确保AI生成内容透明性及防止技术滥用至关重要。本文提出一种基于风格指纹的文本分类新方法,构建由三个不同架构和训练数据的分类器组成的集成模型,用于识别Claude、Gemini、Llama和OpenAI四大知名模型家族生成的文本。为降低误报并提升可信度,仅当三者一致时才认定预测有效。在测试集上,该集成模型达到0.9988的极高精确率和0.0004的极低误报率。此外,模型还能区分已见与未见模型,揭示模型间的潜在风格关联。此方法对验证AI生成文本原创性及追踪模型训练源头具有重要意义。

原文摘要 · Abstract (English)

Large language models (LLMs) have distinct and consistent stylistic fingerprints, even when prompted to write in different writing styles. Detecting these fingerprints is important for many reasons, among them protecting intellectual property, ensuring transparency regarding AI-generated content, and preventing the misuse of AI technologies. In this paper, we present a novel method to classify texts based on the stylistic fingerprints of the models that generated them. We introduce an LLM-detection ensemble that is composed of three classifiers with varied architectures and training data. This ensemble is trained to classify texts generated by four well-known LLM families: Claude, Gemini, Llama, and OpenAI. As this task is highly cost-sensitive and might have severe implications, we want to minimize false-positives and increase confidence. We consider a prediction as valid when all three classifiers in the ensemble unanimously agree on the output classification. Our ensemble is validated on a test set of texts generated by Claude, Gemini, Llama, and OpenAI models, and achieves extremely high precision (0.9988) and a very low false-positive rate (0.0004). Furthermore, we demonstrate the ensemble's ability to distinguish between texts generated by seen and unseen models. This reveals interesting stylistic relationships between models. This approach to stylistic analysis has implications for verifying the originality of AI-generated texts and tracking the origins of model training techniques.

风格识别大模型检测文本溯源

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。