arXiv:2410.04202cs.LG2024-10综述被引 4

用深度迁移学习自动聚合审稿意见并生成元评审,提升论文决策效率

Deep Transfer Learning Based Peer Review Aggregation and Meta-review Generation for Scientific Articles

  • 基于BERT和T5模型,将自然语言审稿意见转化为可计算特征
  • 在真实数据集上,接受决策准确率优于现有模型,元评审生成质量显著提升
  • 适合需要高效处理海量投稿的学术会议或期刊编辑部使用

同行评审是通过一位或多位专家对稿件进行质量评估的过程。作者向学术会议或期刊提交论文,需由同行评审,随后元评审者汇总评审意见并生成元评审与录用决定。近年来投稿量激增,使元评审者在保证质量的前提下及时完成评审面临巨大挑战。本文针对元评审中的两大难题——论文录用决策与元评审生成提出解决方案:首先采用传统机器学习算法结合预训练词嵌入BERT自动预测录用决策;其次提出基于T5的迁移学习模型生成元评审。实验表明,BERT在文本处理中表现优于其他嵌入方法,推荐得分是预测录用的关键特征;微调后的T5在生成任务中优于其他模型。系统以审稿意见及其他相关特征为输入,输出录用判断与元评审内容。统计检验(Wilcoxon符号秩检验)确认新模型在两项任务上均有显著性能提升。

原文摘要 · Abstract (English)

Peer review is the quality assessment of a manuscript by one or more peer experts. Papers are submitted by the authors to scientific venues, and these papers must be reviewed by peers or other authors. The meta-reviewers then gather the peer reviews, assess them, and create a meta-review and decision for each manuscript. As the number of papers submitted to these venues has grown in recent years, it becomes increasingly challenging for meta-reviewers to collect these peer evaluations on time while still maintaining the quality that is the primary goal of meta-review creation. In this paper, we address two peer review aggregation challenges a meta-reviewer faces: paper acceptance decision-making and meta-review generation. Firstly, we propose to automate the process of acceptance decision prediction by applying traditional machine learning algorithms. We use pre-trained word embedding techniques BERT to process the reviews written in natural language text. For the meta-review generation, we propose a transfer learning model based on the T5 model. Experimental results show that BERT is more effective than the other word embedding techniques, and the recommendation score is an important feature for the acceptance decision prediction. In addition, we figure out that fine-tuned T5 outperforms other inference models. Our proposed system takes peer reviews and other relevant features as input to produce a meta-review and make a judgment on whether or not the paper should be accepted. In addition, experimental results show that the acceptance decision prediction system of our task outperforms the existing models, and the meta-review generation task shows significantly improved scores compared to the existing models. For the statistical test, we utilize the Wilcoxon signed-rank test to assess whether there is a statistically significant improvement between paired observations.

同行评审深度学习文本生成自动化决策

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。