arXiv:2412.00281cs.AI2024-12综述被引 3

用AI自动生成论文批注,帮审稿人更快抓住重点。

Streamlining the review process: AI-generated annotations in research manuscripts

  • 用GPT-4自动提取论文关键段落,辅助审稿
  • 9人测试显示工具提升审稿理解与专注度
  • 适合希望提效的期刊编辑和审稿人

科研论文投稿量激增给传统同行评审系统带来巨大压力,导致审稿人工作负担过重。本研究探索将大语言模型(LLMs)融入同行评审流程,以提升效率而不降低质量。聚焦于稿件批注,尤其是内容摘录,作为人机协作的潜在领域。尽管LLMs在覆盖全面性和信息量方面表现优异,但在高层次分析与批判性思维上仍有不足,因此不适合完全替代人类审稿人。本文提出AnnotateGPT平台,利用GPT-4进行论文批注,旨在增强审稿人的理解与专注。通过九名参与者使用技术接受模型(TAM)问卷评估,验证了该工具的有效性,并推广结论。研究强调批注是人机协同的可行中间路径,为将LLMs整合进评审流程提供了洞见,并提出了针对LLM优化的传统批注工具设计方向。

原文摘要 · Abstract (English)

The increasing volume of research paper submissions poses a significant challenge to the traditional academic peer-review system, leading to an overwhelming workload for reviewers. This study explores the potential of integrating Large Language Models (LLMs) into the peer-review process to enhance efficiency without compromising effectiveness. We focus on manuscript annotations, particularly excerpt highlights, as a potential area for AI-human collaboration. While LLMs excel in certain tasks like aspect coverage and informativeness, they often lack high-level analysis and critical thinking, making them unsuitable for replacing human reviewers entirely. Our approach involves using LLMs to assist with specific aspects of the review process. This paper introduces AnnotateGPT, a platform that utilizes GPT-4 for manuscript review, aiming to improve reviewers' comprehension and focus. We evaluate AnnotateGPT using a Technology Acceptance Model (TAM) questionnaire with nine participants and generalize the findings. Our work highlights annotation as a viable middle ground for AI-human collaboration in academic review, offering insights into integrating LLMs into the review process and tuning traditional annotation tools for LLM incorporation.

AI审稿大模型人机协作

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。