评估大模型模仿人类写作风格的能力,发现提示策略比模型大小影响更大。
How Well Do LLMs Imitate Human Writing Style?
- 无需训练,结合词频与嵌入向量判断文本风格相似性
- 跨领域准确率达94.5%,生成文本与原文风格匹配度超99%
- 揭示高仿真的模型仍缺乏人类写作的随机性,适合内容检测研究
大语言模型虽能生成流畅文本,但其复现特定作者写作风格的能力尚不明确。本文提出一种快速、无需训练的作者身份验证与风格模仿分析框架,融合TF-IDF字符n-gram与Transformer嵌入,通过经验距离分布分类文本对,避免监督训练和阈值调优。在学术论文上达到97.5%准确率,跨领域评估达94.5%,相比参数化基线减少91.8%训练时间与59%内存占用。我们评估了三个模型家族(Llama、Qwen、Mixtral)中的五个LLM,采用零样本、单样本、少样本及文本补全四种提示策略。结果表明,提示策略对风格保真度的影响远大于模型规模:少样本提示使风格匹配准确率提升至零样本的23.5倍,补全提示可达99.9%与原作者风格一致。关键发现:高保真模仿并不等于人类般的不可预测性——人类作文平均困惑度为29.5,而匹配的LLM输出仅为15.2。这说明风格保真与统计可检测性可分离,为未来作者建模、识别与身份控制生成提供可复现基础。
原文摘要 · Abstract (English)
Large language models (LLMs) can generate fluent text, but their ability to replicate the distinctive style of a specific human author remains unclear. We present a fast, training-free framework for authorship verification and style imitation analysis. The method integrates TF-IDF character n-grams with transformer embeddings and classifies text pairs through empirical distance distributions, eliminating the need for supervised training or threshold tuning. It achieves 97.5\% accuracy on academic essays and 94.5\% in cross-domain evaluation, while reducing training time by 91.8\% and memory usage by 59\% relative to parameter-based baselines. Using this framework, we evaluate five LLMs from three separate families (Llama, Qwen, Mixtral) across four prompting strategies - zero-shot, one-shot, few-shot, and text completion. Results show that the prompting strategy has a more substantial influence on style fidelity than model size: few-shot prompting yields up to 23.5x higher style-matching accuracy than zero-shot, and completion prompting reaches 99.9\% agreement with the original author's style. Crucially, high-fidelity imitation does not imply human-like unpredictability - human essays average a perplexity of 29.5, whereas matched LLM outputs average only 15.2. These findings demonstrate that stylistic fidelity and statistical detectability are separable, establishing a reproducible basis for future work in authorship modeling, detection, and identity-conditioned generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。