用强化学习训练模型生成更深入、有证据的论文评审问题。
IntelliAsk: Learning to Ask High-Quality Research Questions via RLVR
- 通过人类偏好标注构建奖励模型，指导模型生成高质量问题。
- 生成的问题在专家评估中更扎根论文、更有深度，且减少依赖首页内容。
- 适用于提升大模型评审能力，尤其适合研究者与审稿人使用。
同行评审依赖有实质内容、基于证据的问题，但当前大模型生成的问题多为表面层次，专家评估表现不如人工提问。为此，我们从OpenReview收集高质量评审问题数据，并开展人类偏好研究，由专家从努力程度、证据支持和内容锚定三个维度评估问题-论文配对。基于这些标注，我们训练了IntelliReward——一个基于冻结自回归大模型、结合可训练多头注意力机制的奖励模型。经专家判断验证，IntelliReward在预测问题质量方面优于基于API的SFT基线，具备可扩展性。我们采用解耦剪辑与动态采样策略优化（DAPO）并结合IntelliReward，训练出IntelliAsk，一种对齐人类标准、能生成高努力度、有证据支撑的批判性问题的大模型。人类评估显示，IntelliAsk生成的问题比强基线更扎实、更具深度，且对论文首页内容依赖更少。此外，在推理与写作基准测试中也取得改进，表明评审问题质量与模型综合能力相关。相比Qwen3-32B，IntelliAsk在MuSR（68.3 vs 64.7 准确率）和WritingBench（8.31 vs 8.07 分）上表现更优。我们开源代码、过滤后的评审数据集、专家标注、IntelliAsk及IntelliReward，以支持对大模型生成评审问题的根基性、努力度与证据性进行自动评估。
原文摘要 · Abstract (English)
Peer review relies on substantive, evidence-based questions, yet current LLMs generate surface-level queries that perform worse than human reviewer questions in expert evaluation. To address this gap, we curate a high-quality dataset of reviewer questions from OpenReview and conduct a human preference study where expert annotators evaluate question-paper pairs across three dimensions: effort, evidence, and grounding. From these annotations, we train IntelliReward, a reward model built from a frozen autoregressive LLM with trainable multi-head transformers. Validated against expert judgments, IntelliReward predicts reviewer-question quality better than API-based SFT baselines and provides scalable evaluation. We apply Decoupled Clip and Dynamic Sampling Policy Optimization (DAPO) with IntelliReward to train IntelliAsk, a question-generation model aligned with human standards of effortful, evidence-based critique. Human evaluations show IntelliAsk generates more grounded, substantive and effortful questions than strong baselines and reduces reliance on first-page content. We also find improvements on reasoning and writing benchmarks, suggesting reviewer-question quality correlates with broader capabilities. Compared to Qwen3-32B, IntelliAsk improves MuSR (68.3 vs 64.7 Acc) and WritingBench (8.31 vs 8.07). We release our code, filtered review dataset, expert annotations, IntelliAsk and IntelliReward to support automatic evaluation of grounding, effort, and evidence in LLM-generated review questions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。