研究大模型如何改变科研写作，发现文本更易懂但风格趋同。
What Are LLMs Doing to Scientific Communication? Measuring Changes in Writing Practices and Reading Experience
- 分析3.7万篇论文与3000段改写文本，追踪写作变化
- 大模型使词汇更复杂、句式更固定，多样性下降
- 专家认为内容更易读有趣，但对AI写作存抵触情绪
大语言模型是否改变了科学写作的风格？我们以自然语言处理领域为例，构建了包含超过37,000篇论文（2020–2024年）的自然语料库，以及由3,000段人工撰写文本及其大模型优化版本组成的合成数据集。通过历时性词汇分析，发现词频和使用语境均有显著变化，部分术语出现语义专业化，另一些则趋于泛化。进一步建模复杂文体特征，结果显示经大模型修改的文本更常使用特定句法结构，词汇更长且更复杂，词汇多样性降低。最后，通过20位领域专家的试点标注研究，将写作变化与阅读体验关联：专家普遍认为大模型优化后的文本更易理解、更具吸引力，但也表现出对大模型使用的负面定性态度，凸显人工智能辅助写作对阅读体验的强烈主观影响。
原文摘要 · Abstract (English)
Has the style of scientific communication changed due to the growing use of large language models in the writing process? We address this question in the domain of Natural Language Processing by leveraging two data resources we create: a naturalistic corpus of over 37,000 papers from the ACL Anthology (2020-2024); and a synthetic dataset of 3,000 human-written passages and their LLM-generated improvements. We first implement a series of diachronic lexical analyses, showing that both word frequency and usage contexts have changed significantly over time, indicating semantic specialization in some cases and generalization in others. Broadening our perspective, we then model a range of more complex stylistic features and find that LLM-modified texts more frequently contain certain syntactic constructions, more complex and longer words and a lower lexical diversity. Finally, we connect these changes in writing practices to subjective reading experience through a pilot annotation study with 20 domain experts. They overall rate LLM-improved texts as more understandable and exciting, but also express negative qualitative attitudes towards LLMs, highlighting the strongly subjective effect of AI-assisted writing on reading experience.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。