用审稿回应指导AI生成可执行的评审意见。
RbtAct: Rebuttal as Supervision for Actionable Review Feedback Generation
- 以审稿回应作为隐式监督,训练模型生成具体可操作建议。
- 在RMR-75K数据集上微调,行动力与精准度显著优于基线。
- 适合需要提升评审质量的研究者与期刊编辑使用。
大语言模型被广泛用于科学工作流程中的同行评审报告撰写,但多数AI生成的评审意见流于表面、缺乏可操作性,作者难以获得具体修改指引。本文提出RbtAct,将审稿回应置于学习核心,利用其揭示哪些评论促使了实际修改或具体计划,哪些仅被辩护。基于此,我们构建新任务:视角条件下的段级反馈生成,要求模型根据全文和指定视角(如实验、写作)生成聚焦评论。同时建立大规模数据集RMR-75K,标注评审段落与对应回应段落,并附视角标签与作者采纳影响等级。使用监督微调与基于回应对的偏好优化方法,在Llama-3.1-8B-Instruct模型上进行训练。人类专家与LLM评估均显示,该方法在行动力与具体性方面持续优于强基线,同时保持内容相关性与事实准确性。
原文摘要 · Abstract (English)
Large language models (LLMs) are increasingly used across the scientific workflow, including to draft peer-review reports. However, many AI-generated reviews are superficial and insufficiently actionable, leaving authors without concrete, implementable guidance and motivating the gap this work addresses. We propose RbtAct, which targets actionable review feedback generation and places existing peer review rebuttal at the center of learning. Rebuttals show which reviewer comments led to concrete revisions or specific plans, and which were only defended. Building on this insight, we leverage rebuttal as implicit supervision to directly optimize a feedback generator for actionability. To support this objective, we propose a new task called perspective-conditioned segment-level review feedback generation, in which the model is required to produce a single focused comment based on the complete paper and a specified perspective such as experiments and writing. We also build a large dataset named RMR-75K that maps review segments to the rebuttal segments that address them, with perspective labels and impact categories that order author uptake. We then train the Llama-3.1-8B-Instruct model with supervised fine-tuning on review segments followed by preference optimization using rebuttal derived pairs. Experiments with human experts and LLM-as-a-judge show consistent gains in actionability and specificity over strong baselines while maintaining grounding and relevance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。