测试大模型能否真实模仿文人政客的写作风格,发现仍可被精准识别。
Decoding AI Authorship: Can LLMs Truly Mimic Human Style Across Literature and Politics?
- 用零样本提示生成仿写文本,结合分类与可解释模型评估
- 仅用8个特征就达到神经网络级识别准确率,困惑度是关键判别指标
- 模型能模仿语法和可读性,但无法复制人类情感密度与风格变异
随着生成式AI模仿特定人类风格能力的提升,本研究检验了GPT-4o、Gemini 1.5 Pro和Claude Sonnet 3.5等先进大语言模型(LLMs)在模仿沃尔特·惠特曼、威廉·华兹华斯、唐纳德·特朗普和巴拉克·奥巴马等著名文学与政治人物作者风格方面的能力。采用严格的主题对齐零样本提示框架生成合成语料,并通过结合Transformer分类(BERT)与可解释机器学习(XGBoost)的互补评估体系进行分析。方法融合语言查询与词频(LIWC)标记、困惑度及可读性指数,衡量生成文本与人类写作之间的差异。结果表明,AI生成内容仍高度可检测:基于八个风格特征的XGBoost模型识别准确率接近高维神经分类器水平。特征重要性分析指出,困惑度是主要判别指标,揭示了人工智能输出在随机规律性上与人类写作的显著差异。尽管模型在低维启发式特征(如句法复杂度和可读性)上呈现分布趋同,但尚未完全复现人类语料中固有的情感密度与风格变异性。本研究为当前生成式风格模仿的统计缺口提供了全面基准,对数字人文与社交媒体中的作者归属具有重要意义。
原文摘要 · Abstract (English)
Amidst the rising capabilities of generative AI to mimic specific human styles, this study investigates the ability of state-of-the-art large language models (LLMs), including GPT-4o, Gemini 1.5 Pro, and Claude Sonnet 3.5, to emulate the authorial signatures of prominent literary and political figures: Walt Whitman, William Wordsworth, Donald Trump, and Barack Obama. Utilizing a zero-shot prompting framework with strict thematic alignment, we generated synthetic corpora evaluated through a complementary framework combining transformer-based classification (BERT) and interpretable machine learning (XGBoost). Our methodology integrates Linguistic Inquiry and Word Count (LIWC) markers, perplexity, and readability indices to assess the divergence between AI-generated and human-authored text. Results demonstrate that AI-generated mimicry remains highly detectable, with XGBoost models trained on a restricted set of eight stylometric features achieving accuracy comparable to high-dimensional neural classifiers. Feature importance analyses identify perplexity as the primary discriminative metric, revealing a significant divergence in the stochastic regularity of AI outputs compared to the higher variability of human writing. While LLMs exhibit distributional convergence with human authors on low-dimensional heuristic features, such as syntactic complexity and readability, they do not yet fully replicate the nuanced affective density and stylistic variance inherent in the human-authored corpus. By isolating the specific statistical gaps in current generative mimicry, this study provides a comprehensive benchmark for LLM stylistic behavior and offers critical insights for authorship attribution in the digital humanities and social media.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。