arXiv:2510.13201cs.CVcs.AI2025-10综述被引 9

构建论文评审演化追踪系统,揭示人工智能会议审稿变迁规律。

Paper Copilot: Tracking the Evolution of Peer Review in AI Conferences

  • 建立跨会议的评审数字档案与开放数据集
  • 分析多届ICLR评审,发现审稿标准演变趋势
  • 助力社区透明化改进学术评审机制

人工智能会议的快速扩张正加剧本已脆弱的同行评审体系压力,导致审稿人负担过重、专业匹配度低、评价标准不一、评审内容浅显或模板化,以及在压缩时间下的问责缺失。为应对这一挑战,会议组织方虽推出多项新政策与干预措施,但这些临时性调整反而引发更多困惑,使论文最终录用情况及评审实践的年度演变过程仍不透明。我们提出Paper Copilot,一个可持久保存多个计算机科学领域会议同行评审记录的系统,构建了一个开放数据集,支持大规模研究同行评审。通过发布基础设施与数据集,该系统推动对评审演化过程的可复现研究。我们希望这些资源能帮助学术界追踪变化、诊断问题,并基于证据推动更稳健、透明、可靠的评审体系发展。

原文摘要 · Abstract (English)

The rapid growth of AI conferences is straining an already fragile peer-review system, leading to heavy reviewer workloads, expertise mismatches, inconsistent evaluation standards, superficial or templated reviews, and limited accountability under compressed timelines. In response, conference organizers have introduced new policies and interventions to preserve review standards. Yet these ad-hoc changes often create further concerns and confusion about the review process, leaving how papers are ultimately accepted - and how practices evolve across years - largely opaque. We present Paper Copilot, a system that creates durable digital archives of peer reviews across a wide range of computer-science venues, an open dataset that enables researchers to study peer review at scale, and a large-scale empirical analysis of ICLR reviews spanning multiple years. By releasing both the infrastructure and the dataset, Paper Copilot supports reproducible research on the evolution of peer review. We hope these resources help the community track changes, diagnose failure modes, and inform evidence-based improvements toward a more robust, transparent, and reliable peer-review system.

同行评审学术生态数据开放

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。