用轻量模型高效提取关键引用，提升大模型问答准确率
LLMQuoter: Enhancing RAG Capabilities Through Efficient Quote Extraction From Large Contexts
- 先抽关键句子再推理，降低大模型负担
- 在热锅问答数据集上比全上下文方法高20多分
- 适合想快速提升RAG性能的开发者和研究者
我们提出LLMQuoter，一种基于知识蒸馏的轻量级模型，用于增强检索增强生成（RAG）能力。该模型基于LLaMA-3B架构，使用HotpotQA的15,000个样本子集，通过低秩适配（LoRA）微调，采用“先引述后回答”策略，在长上下文中高效识别关键证据片段，再传递给推理模型。此流程显著降低认知负荷，相比全上下文方法如检索增强微调（RAFT），在小模型和大模型上均实现超过20点的准确率提升。通过从高性能教师模型中蒸馏知识，LLMQuoter在资源受限的微调设置下仍表现优异，使高级RAG能力更易获取，无需大规模重训练。结果表明，基于抽取的推理可有效简化复杂流程，为研究者与实践者提供可扩展、实用的解决方案。
原文摘要 · Abstract (English)
We introduce LLMQuoter, a lightweight, distillation-based model designed to enhance Retrieval Augmented Generation (RAG) by extracting the most relevant textual evidence for downstream reasoning tasks. Built on the LLaMA-3B architecture and fine-tuned with Low-Rank Adaptation (LoRA) on a 15,000-sample subset of HotpotQA, LLMQuoter adopts a "quote-first-then-answer" strategy, efficiently identifying key quotes before passing curated snippets to reasoning models. This workflow reduces cognitive overhead and outperforms full-context approaches like Retrieval-Augmented Fine-Tuning (RAFT), achieving over 20-point accuracy gains across both small and large language models. By leveraging knowledge distillation from a high-performing teacher model, LLMQuoter achieves competitive results in a resource-efficient fine-tuning setup. It democratizes advanced RAG capabilities, delivering significant performance improvements without requiring extensive model retraining. Our results highlight the potential of distilled quote-based reasoning to streamline complex workflows, offering a scalable and practical solution for researchers and practitioners alike.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。