arXiv:2412.01708cs.CLcs.AI2024-12综述被引 59

LLM生成的审稿意见易被操纵,可能扭曲学术评审公正性。

Are We There Yet? Revealing the Risks of Utilizing Large Language Models in Scholarly Peer Review

  • 通过隐藏内容注入和刻意强调缺陷,可操控LLM审稿结果。
  • 仅5%审稿被操纵就可能导致12%论文排名下滑至前30%以下。
  • 适合关注AI伦理与学术评审可信度的研究者阅读。

学术同行评审是科学进步的核心,但投稿量激增与流程繁重使其面临压力。近年来,大语言模型(LLMs)被引入评审环节,其生成意见与人类高度重合,前景可观。然而,未经管控的使用存在重大风险。本研究系统分析了LLM评审的脆弱性,包括显式与隐式操纵。实验显示,向稿件中注入隐蔽内容可使LLM评分虚高,且与人类评审一致性下降。模拟表明,操纵5%的评审可能致使12%的论文失去前30%排名。隐式操纵中,作者若刻意突出小缺陷,LLM对其响应一致性比人类高4.5倍。此外,LLM存在固有缺陷:可能更倾向不完整稿件,且在单盲评审中偏好知名作者。这些发现警示我们,当前尚不具备广泛采用LLM进行评审的条件,亟需建立强健防护机制。

原文摘要 · Abstract (English)

Scholarly peer review is a cornerstone of scientific advancement, but the system is under strain due to increasing manuscript submissions and the labor-intensive nature of the process. Recent advancements in large language models (LLMs) have led to their integration into peer review, with promising results such as substantial overlaps between LLM- and human-generated reviews. However, the unchecked adoption of LLMs poses significant risks to the integrity of the peer review system. In this study, we comprehensively analyze the vulnerabilities of LLM-generated reviews by focusing on manipulation and inherent flaws. Our experiments show that injecting covert deliberate content into manuscripts allows authors to explicitly manipulate LLM reviews, leading to inflated ratings and reduced alignment with human reviews. In a simulation, we find that manipulating 5% of the reviews could potentially cause 12% of the papers to lose their position in the top 30% rankings. Implicit manipulation, where authors strategically highlight minor limitations in their papers, further demonstrates LLMs' susceptibility compared to human reviewers, with a 4.5 times higher consistency with disclosed limitations. Additionally, LLMs exhibit inherent flaws, such as potentially assigning higher ratings to incomplete papers compared to full papers and favoring well-known authors in single-blind review process. These findings highlight the risks of over-reliance on LLMs in peer review, underscoring that we are not yet ready for widespread adoption and emphasizing the need for robust safeguards.

LLM风险学术评审AI伦理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。