用审稿人评分提升论文评价准确率，比仅看作者背景更有效
From Authors to Reviewers: Leveraging Rankings to Improve Peer Review
- 利用审稿人打分排名优化论文评估，替代传统作者背景依赖
- 结合审稿人与作者排名可显著提升论文质量判断准确性
- 适用于关注评审公平性的机器学习会议组织者与研究者
本文讨论了Su等（2025）在2025年《JASA》发表的论文。我们祝贺作者对2023年ICML投稿数据进行了全面而深刻的实证研究。近年来，随着提交论文数量迅速增加，机器学习会议的评审质量成为重要关切。本文提出一种替代方案：利用审稿人的排名信息，而非作者的排名信息。我们模拟了接近2023年ICML投稿情况的评审数据。结果表明：(i) 引入审稿人排名信息能显著提升对每篇论文质量的评估，通常优于仅使用作者排名；(ii) 将审稿人和作者的排名信息相结合，在多数场景下可实现最准确的论文评估。
原文摘要 · Abstract (English)
This paper is a discussion of the 2025 JASA discussion paper by Su et al. (2025). We would like to congratulate the authors on conducting a comprehensive and insightful empirical investigation of the 2023 ICML ranking data. The review quality of machine learning (ML) conferences has become a big concern in recent years, due to the rapidly growing number of submitted manuscripts. In this discussion, we propose an approach alternative to Su et al. (2025) that leverages ranking information from reviewers rather than authors. We simulate review data that closely mimics the 2023 ICML conference submissions. Our results show that (i) incorporating ranking information from reviewers can significantly improve the evaluation of each paper's quality, often outperforming the use of ranking information from authors alone; and (ii) combining ranking information from both reviewers and authors yields the most accurate evaluation of submitted papers in most scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。