用对话模型辅助论文评审决策，提升效率与准确性。
Decision-Making with Deliberation: Meta-reviewing as a Document-grounded Dialogue
- 将元评审视为对话式决策过程，构建针对性对话代理。
- 利用大模型生成高质量合成数据，解决训练数据稀缺问题。
- 在真实评审场景中验证效果，显著提升元评审效率。
元评审是同行评审流程中的关键环节，决定论文是否被接受。以往研究将其视为对审稿意见的摘要任务，但实际是一个需要权衡各方论据并置于整体背景下的决策过程。已有研究表明，对话代理可有效辅助此类决策。本文针对实现元评审对话代理的实际挑战展开研究：首先，通过基于自精炼策略的大语言模型生成合成数据，缓解训练数据稀缺问题，实验证明该方法生成的对话更贴近专家领域；其次，利用该数据训练专用于元评审的对话代理，其性能优于现成的大模型助手；最后，在真实元评审场景中应用，验证了其在提升评审效率方面的有效性。
原文摘要 · Abstract (English)
Meta-reviewing is a pivotal stage in the peer-review process, serving as the final step in determining whether a paper is recommended for acceptance. Prior research on meta-reviewing has treated this as a summarization problem over review reports. However, complementary to this perspective, meta-reviewing is a decision-making process that requires weighing reviewer arguments and placing them within a broader context. Prior research has demonstrated that decision-makers can be effectively assisted in such scenarios via dialogue agents. In line with this framing, we explore the practical challenges for realizing dialog agents that can effectively assist meta-reviewers. Concretely, we first address the issue of data scarcity for training dialogue agents by generating synthetic data using Large Language Models (LLMs) based on a self-refinement strategy to improve the relevance of these dialogues to expert domains. Our experiments demonstrate that this method produces higher-quality synthetic data and can serve as a valuable resource towards training meta-reviewing assistants. Subsequently, we utilize this data to train dialogue agents tailored for meta-reviewing and find that these agents outperform \emph{off-the-shelf} LLM-based assistants for this task. Finally, we apply our agents in real-world meta-reviewing scenarios and confirm their effectiveness in enhancing the efficiency of meta-reviewing.\footnote{Code available at: https://github.com/UKPLab/eacl2026-meta-review-as-dialog
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。