解决长文档问答中信息丢失问题，双视角增强检索生成效果
LongRAG: A Dual-Perspective Retrieval-Augmented Generation Paradigm for Long-Context Question Answering
- 设计双视角检索机制，兼顾全局信息与细节事实
- 在三个多跳数据集上提升显著，最高超越基线17.25%
- 可插拔架构，适配多种模型与领域使用
长上下文问答（LCQA）旨在对长篇文档进行推理以生成准确答案。现有长上下文大语言模型常遭遇“中间信息丢失”问题。检索增强生成（RAG）通过提供外部事实证据缓解此问题，但其分块策略破坏了全局信息连贯性，且长文本中低质量检索导致噪声干扰，使模型难以识别有效事实。为此，我们提出LongRAG，一种通用、双视角、鲁棒的基于大模型的RAG范式，用于提升对复杂长上下文知识（即全局信息与事实细节）的理解能力。LongRAG设计为即插即用架构，便于适配不同领域和模型。在三个多跳数据集上的大量实验表明，LongRAG显著优于长上下文LLM（提升6.94%）、先进RAG（提升6.16%）和原始RAG（提升17.25%）。此外，通过定量消融实验与多维度分析，验证了系统组件与微调策略的有效性。数据与代码已公开于https://github.com/QingFei1/LongRAG。
原文摘要 · Abstract (English)
Long-Context Question Answering (LCQA), a challenging task, aims to reason over long-context documents to yield accurate answers to questions. Existing long-context Large Language Models (LLMs) for LCQA often struggle with the "lost in the middle" issue. Retrieval-Augmented Generation (RAG) mitigates this issue by providing external factual evidence. However, its chunking strategy disrupts the global long-context information, and its low-quality retrieval in long contexts hinders LLMs from identifying effective factual details due to substantial noise. To this end, we propose LongRAG, a general, dual-perspective, and robust LLM-based RAG system paradigm for LCQA to enhance RAG's understanding of complex long-context knowledge (i.e., global information and factual details). We design LongRAG as a plug-and-play paradigm, facilitating adaptation to various domains and LLMs. Extensive experiments on three multi-hop datasets demonstrate that LongRAG significantly outperforms long-context LLMs (up by 6.94%), advanced RAG (up by 6.16%), and Vanilla RAG (up by 17.25%). Furthermore, we conduct quantitative ablation studies and multi-dimensional analyses, highlighting the effectiveness of the system's components and fine-tuning strategies. Data and code are available at https://github.com/QingFei1/LongRAG.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。