对比两种方法，评估AI生成内容的作者归属准确性。
Attribution Quality in AI-Generated Content:Benchmarking Style Embeddings and LLM Judges
- 用风格嵌入与LLM裁判分别判断文本来源。
- 对GPT生成文本，风格嵌入准确率达82%，优于LLM裁判的68%。
- 在小说和学术写作中，LLM裁判表现更优，适合语义分析场景。
在大语言模型时代，机器生成文本日益逼近人类写作，作者归属变得愈发困难。本文在包含600个平衡样本的Human AI Parallel Corpus数据集上，基准测试两种互补的归属机制：固定风格嵌入与指令微调的LLM裁判（GPT-4o）。该数据集覆盖学术、新闻、小说、博客、口语对话及影视剧本六类，每条数据包含人工提示及对应的黄金续写和由GPT-4o或LLaMA-70B-Instruct生成的续写。结果显示，风格嵌入在识别GPT生成文本时整体准确率达82%，优于LLM裁判的68%；而在识别LLaMA生成文本时，LLM裁判略胜一筹（85% vs. 81%），但差异不显著。关键的是，LLM裁判在小说和学术文本中表现显著更佳，体现其语义敏感性；而风格嵌入在口语和剧本类对话中占优，反映其结构识别优势。二者互补，表明作者归属是多维度问题，需融合策略。为保障可复现性，代码已开源于GitHub，衍生数据发布于Hugging Face，均采用MIT许可。本研究提供了一个可复现的评估框架，并回顾了相关文献。
原文摘要 · Abstract (English)
Attributing authorship in the era of large language models (LLMs) is increasingly challenging as machine-generated prose rivals human writing. We benchmark two complementary attribution mechanisms , fixed Style Embeddings and an instruction-tuned LLM judge (GPT-4o) on the Human AI Parallel Corpus, an open dataset of 600 balanced instances spanning six domains (academic, news, fiction, blogs, spoken transcripts, and TV/movie scripts). Each instance contains a human prompt with both a gold continuation and an LLM-generated continuation from either GPT-4o or LLaMA-70B-Instruct. The Style Embedding baseline achieves stronger aggregate accuracy on GPT continuations (82 pct vs. 68 pct). The LLM Judge is slightly better than the Style embeddings on LLaMA continuations (85 pct vs. 81 pct) but the results are not statistically significant. Crucially, the LLM judge significantly outperforms in fiction and academic prose, indicating semantic sensitivity, whereas embeddings dominate in spoken and scripted dialogue, reflecting structural strengths. These complementary patterns highlight attribution as a multidimensional problem requiring hybrid strategies. To support reproducibility we provide code on GitHub and derived data on Hugging Face under the MIT license. This open framework provides a reproducible benchmark for attribution quality assessment in AI-generated content, along with a review of related literature influencing this work.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。