arXiv:2603.18161cs.CLcs.AI2026-03被引 19

LLMs让写作更中立,改变原意,影响科研评审公正性

How LLMs Distort Our Written Language

  • 用户频繁使用LLM导致文章倾向中立,创作力下降
  • 即使仅要求语法修改,LLM也会显著改变文本语义
  • 生成的论文评审更看重形式而非内容,评分偏高

大型语言模型(LLMs)被全球超十亿人用于写作辅助。本文发现,LLMs不仅改变写作风格和语气,还会持续改变原文意图。首先,通过人类用户研究发现,频繁使用LLM者撰写的论文中,近70%对主题问题的回答趋于中立;大量重度使用者认为写作缺乏创意且非本人风格。其次,基于2021年未受LLM影响的人类作文数据集,我们测试了在专家反馈下让LLM修订文本的效果,结果发现即便仅要求语法修正,LLM仍会显著改变语义。最后,分析某顶级人工智能会议中21%的AI生成同行评审,发现其对研究清晰度与重要性权重更低,平均评分高出一整分。这些发现揭示了公众认知中的AI益处与实际语义扭曲之间的不匹配,警示未来需关注广泛使用AI写作对文化与科学体制的深远影响。

原文摘要 · Abstract (English)

Large language models (LLMs) are used by over a billion people globally, most often to assist with writing. In this work, we demonstrate that LLMs not only alter the voice and tone of human writing but also consistently alter the intended meaning. First, we conduct a human user study to understand how people actually interact with LLMs when using them for writing. Our findings reveal that extensive LLM use led to a nearly 70% increase in essays that remained neutral in answering the topic question. Significantly more heavy LLM users reported that the writing was less creative and not in their voice. Next, using a dataset of human-written essays that was collected in 2021 before the widespread release of LLMs, we study how asking an LLM to revise the essay based on the human-written feedback in the dataset induces large changes in the resulting content and meaning. We find that even when LLMs are prompted with expert feedback and asked to only make grammar edits, they still change the text in a way that significantly alters its semantic meaning. We then examine LLM-generated text in the wild, specifically focusing on the 21% of AI-generated scientific peer reviews at a recent top AI conference. We find that LLM-generated reviews place significantly less weight on clarity and significance of the research, and assign scores that, on average, are a full point higher. These findings highlight a misalignment between the perceived benefit of AI use and an implicit, consistent effect on the semantics of human writing, motivating future work on how widespread AI writing will affect our cultural and scientific institutions.

语言模型写作辅助语义扭曲学术评审

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。