用四步流程自动写学术反驳,效果超人类平均水平
DRPG (Decompose, Retrieve, Plan, Generate): An Agentic Framework for Academic Rebuttal
- 分解评审意见、检索论文证据、规划策略、生成回应,四步闭环
- 规划模块准确率达98%以上,8B模型表现超越普通人类
- 支持多轮复杂反驳,可解释且提供多角度建议
尽管大语言模型在科研流程中应用日益广泛,但针对学术反驳这一关键环节的自动化支持仍严重不足。现有方法多依赖通用大模型或简单流水线,难以处理长文本理解,常生成缺乏针对性与说服力的回复。本文提出DRPG框架,通过四个步骤实现自动学术反驳:分解评审意见为基本关切点,从论文中检索相关证据,规划反驳策略,最终生成响应。其中,规划模块在识别最可行反驳方向时准确率超过98%。在顶级会议数据上的实验表明,DRPG显著优于现有反驳流水线,在仅使用80亿参数模型的情况下达到甚至超越平均人类水平。分析进一步验证了规划设计的有效性,能提供多视角且可解释的建议。此外,该框架在更复杂的多轮反驳场景中也表现良好。这些结果凸显了DRPG的有效性,有望提升学术讨论的质量与规模。代码已开源:https://github.com/ulab-uiuc/DRPG-RebuttalAgent。
原文摘要 · Abstract (English)
Despite the growing adoption of large language models (LLMs) in scientific research workflows, automated support for academic rebuttal, a crucial step in academic communication and peer review, remains largely underexplored. Existing approaches typically rely on off-the-shelf LLMs or simple pipelines, which struggle with long-context understanding and often fail to produce targeted and persuasive responses. In this paper, we propose DRPG, an agentic framework for automatic academic rebuttal generation that operates through four steps: Decompose reviews into atomic concerns, Retrieve relevant evidence from the paper, Plan rebuttal strategies, and Generate responses accordingly. Notably, the Planner in DRPG reaches over 98% accuracy in identifying the most feasible rebuttal direction. Experiments on data from top-tier conferences demonstrate that DRPG significantly outperforms existing rebuttal pipelines and achieves performance beyond the average human level using only an 8B model. Our analysis further demonstrates the effectiveness of the planner design and its value in providing multi-perspective and explainable suggestions. We also showed that DRPG works well in a more complex multi-round setting. These results highlight the effectiveness of DRPG and its potential to provide high-quality rebuttal content and support the scaling of academic discussions. Codes for this work are available at https://github.com/ulab-uiuc/DRPG-RebuttalAgent.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。