用机器学习优化天文项目评审人分配，匹配度提升51个百分点。
Enhancing Peer Review in Astronomy: A Machine Learning and Optimization Approach to Reviewer Assignments for ALMA
- 通过主题建模分析提案与评审人过往提交记录，量化专业匹配度。
- 评审匹配度中位数提升51个百分点，超20%评审人表示自己熟悉分配课题。
- 零需重分提案，节省3至5天人工工作，适合大型科研项目评审系统参考。
随着论文与提案数量持续增长，自动化管理成为应对规模挑战的迫切需求。本研究针对2023年发布的ALMA第10周期提案征集，部署并评估了机器学习与优化算法相结合的评审人分配方法。利用主题建模识别提案主题，并基于评审人过往的ALMA提案记录评估其专业能力。采用改进版PeerReview4All（Stelmakh et al. 2021）分配优化算法，最大化提案主题与评审人专长的匹配度。评估显示，提案主题与评审人专长间的相似度中位数较上一周期提升51个百分点，报告具备相关专长的评审人比例上升20个百分点。且无任何提案因严重不匹配而需重新分配，节省3至5天人工处理时间。
原文摘要 · Abstract (English)
The increasing volume of papers and proposals that undergo peer review emphasizes the pressing need for greater automation to effectively manage the growing scale. In this study, we present the deployment and evaluation of machine learning and optimization techniques to assign proposals to reviewers that were developed for the Atacama Large Millimeter/submillimeter Array (ALMA) during the Cycle 10 Call for Proposals issued in 2023. Using topic modeling algorithms, we identify the proposal topics and assess reviewers' expertise based on their previous ALMA proposal submissions. We then apply an adapted version of the assignment optimization algorithm from PeerReview4All (Stelmakh et al. 2021) to maximize the alignment between proposal topics and reviewer expertise. Our evaluation shows a significant improvement in matching reviewer expertise: the median similarity score between the proposal topic and reviewer expertise increased by 51 percentage points compared to the previous cycle, and the percentage of reviewers reporting expertise in their assigned proposals rose by 20 percentage points. Furthermore, the assignment process proved highly effective in that no proposals required reassignment due to significant mismatches, resulting in a savings of 3 to 5 days of manual effort.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。