AI生成的科研论文有24%存在抄袭，且能躲过检测。
All That Glitters is Not Novel: Plagiarism in AI Generated Research
- 13位专家评估50篇AI生成论文，发现24%存在直接剽窃或改写。
- 76%的论文与已有研究存在不同程度相似，仅少数真正新颖。
- 现有查重工具无法有效识别此类智能抄袭，需人工审慎评估。
自动化科学研发被视为科学的终极前沿。近期多项研究宣称自主科研代理可生成新颖研究思路。然而，在乐观氛围中，我们揭示了一个关键问题：大量此类科研文档实为巧妙剽窃。不同于以往由专家评估研究思路的新颖性与可行性，我们让13位专家在不同情境下评估大模型生成的研究文档与已有工作的相似性。令人担忧的是，专家识别出50篇被评估文档中有24%存在改写（具有一一对应的方法映射）或显著借鉴，这些情况已由原作者交叉验证。其余76%的文档与已有工作存在不同程度的相似性，仅有极小部分表现完全新颖。更严重的是，这些由大模型生成的科研文档未标注原始来源，且能绕过内置的抄袭检测机制。通过受控实验，我们进一步证明自动化抄袭检测系统对这类系统的剽窃行为识别能力不足。我们建议对大模型生成的研究进行审慎评估，并讨论了研究结果对学术出版的影响。
原文摘要 · Abstract (English)
Automating scientific research is considered the final frontier of science. Recently, several papers claim autonomous research agents can generate novel research ideas. Amidst the prevailing optimism, we document a critical concern: a considerable fraction of such research documents are smartly plagiarized. Unlike past efforts where experts evaluate the novelty and feasibility of research ideas, we request $13$ experts to operate under a different situational logic: to identify similarities between LLM-generated research documents and existing work. Concerningly, the experts identify $24\%$ of the $50$ evaluated research documents to be either paraphrased (with one-to-one methodological mapping), or significantly borrowed from existing work. These reported instances are cross-verified by authors of the source papers. The remaining $76\%$ of documents show varying degrees of similarity with existing work, with only a small fraction appearing completely novel. Problematically, these LLM-generated research documents do not acknowledge original sources, and bypass inbuilt plagiarism detectors. Lastly, through controlled experiments we show that automated plagiarism detectors are inadequate at catching plagiarized ideas from such systems. We recommend a careful assessment of LLM-generated research, and discuss the implications of our findings on academic publishing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。