让大模型分步规划搜索,提升问答准确率与效率。
Distilling Reasoning Without Knowledge: A Framework for Reliable LLMs
- 分离规划与检索,用轻量级学生模型学结构化推理步骤
- 在SEAL-0上准确率提升,推理延迟更低
- 适合需要高可靠性的实时问答场景
基于大语言模型的事实类问答在依赖最新或冲突信息时仍不可靠。尽管检索增强和工具调用模型减少了幻觉,但常依赖隐式规划,导致工具使用低效。本文提出一种模块化框架,显式分离规划、事实检索与答案生成。通过教师-学生训练,轻量级学生规划器学习生成包含抽象推理步骤和可搜索事实请求的结构化分解。监督信号仅包含规划轨迹与事实请求,不提供答案或检索证据。推理时,规划器生成计划,提示工程模块完成检索与应答合成。在极端挑战性的SEAL-0基准测试中,该框架相比单体推理模型和提示驱动工具框架,显著提升准确率并降低延迟,证明显式学习的规划结构对可靠的事实查询大模型至关重要。
原文摘要 · Abstract (English)
Fact-seeking question answering with large language models (LLMs) remains unreliable when answers depend on up-to-date or conflicting information. Although retrieval-augmented and tool-using LLMs reduce hallucinations, they often rely on implicit planning, leading to inefficient tool usage. We propose a modular framework that explicitly separates planning from factual retrieval and answer synthesis. A lightweight student planner is trained via a teacher-student framework to generate structured decompositions consisting of abstract reasoning steps and searchable fact requests. The supervision signals contain only planning traces and fact requests, without providing factual answers or retrieved evidence. At inference, the planner produces plans, while prompt-engineered modules perform retrieval and response synthesis. We evaluate the proposed framework on SEAL-0, an extremely challenging benchmark for search-augmented LLMs. Results show that supervised planning improves both accuracy and latency compared to monolithic reasoning models and prompt-based tool-augmented frameworks, demonstrating that explicitly learned planning structures are essential for reliable fact-seeking LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。