对比人类与大模型文本，发现人类写作更富变化和情感。
Human Variability vs. Machine Consistency: A Linguistic Analysis of Texts Generated by Humans and Large Language Models
- 用250个语言特征分析文本，发现人类写作变异性更高。
- 人类文本平均句法深度更低，语义内容更丰富，情感表达更强烈。
- 适合关注生成文本质量、人机差异的研究者参考。
大语言模型（LLMs）生成的文本日益接近人类写作，现有研究多聚焦于区分文本来源。本文基于四个领域的文本，利用LFTK工具自动计算250个语言特征，并额外测量每篇文档的平均句法深度、语义相似度和情感内容。通过二维PCA降维分析发现，人类写作在各项特征上的变异性显著高于大模型生成文本，尤其在语言风格约束较弱的文体中更为明显。人类文本整体认知负荷较低，语义内容更丰富，情感表达更充沛。这些结果表明，需引入有意义的语言特征以深化对大模型输出的理解。
原文摘要 · Abstract (English)
The rapid advancements in large language models (LLMs) have significantly improved their ability to generate natural language, making texts generated by LLMs increasingly indistinguishable from human-written texts. Recent research has predominantly focused on using LLMs to classify text as either human-written or machine-generated. In our study, we adopt a different approach by profiling texts spanning four domains based on 250 distinct linguistic features. We select the M4 dataset from the Subtask B of SemEval 2024 Task 8. We automatically calculate various linguistic features with the LFTK tool and additionally measure the average syntactic depth, semantic similarity, and emotional content for each document. We then apply a two-dimensional PCA reduction to all the calculated features. Our analyses reveal significant differences between human-written texts and those generated by LLMs, particularly in the variability of these features, which we find to be considerably higher in human-written texts. This discrepancy is especially evident in text genres with less rigid linguistic style constraints. Our findings indicate that humans write texts that are less cognitively demanding, with higher semantic content, and richer emotional content compared to texts generated by LLMs. These insights underscore the need for incorporating meaningful linguistic features to enhance the understanding of textual outputs of LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。