LLM让科研产出翻倍，但质量参差，需重新评估学术成果。
Scientific production in the era of Large Language Models
- 用LLM写论文可使发文量提升23.7%至89.3%。
- 语言复杂但内容空洞的论文增多，质量与文风脱钩。
- 使用LLM者更广泛引用新书和低被引文献，视野更广。
大型语言模型（LLMs）正在迅速重塑科学研发。我们基于包含210万篇预印本、2.8万份同行评审报告及2.46亿次科学文档访问记录的多大规模数据集进行分析。发现：1）采用LLM撰写稿件的科学家论文产出量显著上升，增幅在23.7%至89.3%之间，具体取决于学科领域和作者背景；2）LLM使用逆转了写作复杂度与论文质量之间的关系，导致大量语言繁复但内容薄弱的稿件涌现；3）使用LLM的研究者更倾向于查阅和引用多样化前期成果，包括书籍以及较新且被引次数较低的文献。这些发现揭示了科研生产模式的重大转变，可能需要期刊、资助机构及职称评定委员会重新审视科研评价体系。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are rapidly reshaping scientific research. We analyze these changes in multiple, large-scale datasets with 2.1M preprints, 28K peer review reports, and 246M online accesses to scientific documents. We find: 1) scientists adopting LLMs to draft manuscripts demonstrate a large increase in paper production, ranging from 23.7-89.3% depending on scientific field and author background, 2) LLM use has reversed the relationship between writing complexity and paper quality, leading to an influx of manuscripts that are linguistically complex but substantively underwhelming, and 3) LLM adopters access and cite more diverse prior work, including books and younger, less-cited documents. These findings highlight a stunning shift in scientific production that will likely require a change in how journals, funding agencies, and tenure committees evaluate scientific works.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。