用AI给科研论文提反馈,让全球研究者都能更平等地获得学术批评。
Human-AI Collaboration in Science at Scale: A Global Large-scale Randomized Field Experiment
- 在3.1万篇论文上测试定制化AI反馈,随机分组对比效果。
- 收到AI反馈的作者修改论文概率提升12.55%,尤其对非英语区作者更明显。
- 帮助了早期学者和低影响力团队,有望推动科研公平与效率。
合作是现代科学的核心模式,但其关键机制——反馈——难以观测、难于规模化且分布不均。本文通过一项全球大规模随机实地实验,为超过31,000篇arXiv预印本(覆盖150个领域,45,000多名来自133个地区的研究人员)提供了定制化的大型语言模型(LLM)生成反馈。相比对照组,接收到反馈的作者修订稿件的可能性显著提高,相对基线修订率提升了12.55%。接触AI反馈还促使作者在后续论文中更频繁使用LLM工具,表明其对科研习惯有长期影响。该效应在非英语主导地区作者、文献嵌入度较低的论文以及低h指数、早期职业阶段的团队中尤为显著,说明AI反馈可能在原本获取及时批判资源受限的群体中带来最大价值。这些结果提供了因果证据:结构化的基于AI的干预能将科学反馈这一稀缺资源从少数人独占转变为更广泛可及的公共品,对全球科研体系的生产力、公平性与能力发展具有深远意义。
原文摘要 · Abstract (English)
Collaboration is the defining mode of modern science, yet its core mechanism -- feedback -- remains hard to observe, difficult to scale, and unequally distributed. Here we test whether large language models (LLMs) can contribute to this hidden but vital practice and reallocate scientific feedback, an essential yet scarce resource for knowledge production. In a global large-scale randomized field experiment, we delivered customized LLM-generated feedback for over 31,000 arXiv preprints across 150 fields and more than 45,000 researchers from 133 geographic regions. Relative to controls, authors who received feedback had a significantly higher likelihood of revising their manuscripts, corresponding to a 12.55% relative increase over the baseline revision rate. Exposure to AI feedback also increased authors' subsequent use of LLM tools in their future papers, suggesting longer-run shifts in scientific practice. These effects were strongest among authors from non-English-dominant research regions, manuscripts less embedded in the scholarly literature, and teams with lower h-indexes and earlier career stages, consistent with the idea that AI feedback may provide the greatest benefit where access to timely critique is otherwise limited. Together, these findings provide causal evidence that structured AI-based interventions can transform access to scientific feedback from a largely private advantage into a more widely distributed resource, with broader implications for productivity, equity, and capacity across the global research system.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。