让小模型读论文更高效,通过分步规划提升准确率
PaperGuide: Making Small Language-Model Paper-Reading Agents More Efficient
- 将阅读任务拆解为计划与执行两阶段,减少盲目尝试
- 在论文问答任务中效率更高,性能媲美大模型
- 适合想用小模型做科研辅助的研究者
科学文献的爆炸式增长使得研究人员仅靠人工阅读难以跟上进展。近年来,大语言模型(LLMs)的发展推动了自主阅读代理的兴起,但现有方法多依赖复杂提示工程或传统SFT-RL训练流程,导致探索成本高、收益低。受认知科学启发,本文提出PaperCompass框架,将高层规划与细粒度执行分离:先生成显式行动序列计划,再逐项细化为函数调用参数。为此设计了专用于该框架的强化学习方法Draft-and-Follow Policy Optimization(DFPO),联合优化计划与最终解决方案。理论分析表明,DFPO具备稳定训练的优化性质。在Paper-QA基准测试中,PaperCompass在不牺牲性能的前提下显著提升效率,表现接近更大规模模型。
原文摘要 · Abstract (English)
The accelerating growth of the scientific literature makes it increasingly difficult for researchers to track new advances through manual reading alone. Recent progress in large language models (LLMs) has therefore spurred interest in autonomous agents that can read scientific papers and extract task-relevant information. However, most existing approaches rely either on heavily engineered prompting or on a conventional SFT-RL training pipeline, both of which often lead to excessive and low-yield exploration. Drawing inspiration from cognitive science, we propose PaperCompass, a framework that mitigates these issues by separating high-level planning from fine-grained execution. PaperCompass first drafts an explicit plan that outlines the intended sequence of actions, and then performs detailed reasoning to instantiate each step by selecting the parameters for the corresponding function calls. To train such behavior, we introduce Draft-and-Follow Policy Optimization (DFPO), a tailored RL method that jointly optimizes both the draft plan and the final solution. DFPO can be viewed as a lightweight form of hierarchical reinforcement learning, aimed at narrowing the `knowing-doing' gap in LLMs. We provide a theoretical analysis that establishes DFPO's favorable optimization properties, supporting a stable and reliable training process. Experiments on paper-based question answering (Paper-QA) benchmarks show that PaperCompass improves efficiency over strong baselines without sacrificing performance, achieving results comparable to much larger models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。