arXiv:2410.15277cs.CL2024-10NAACL被引 8

用压缩文档摘要提升多跳问答效率,降低延迟。

BRIEF: Bridging Retrieval and Inference for Multi-hop Reasoning via Compression

  • 将检索文档压缩成高密度摘要,融入上下文增强推理
  • 压缩率提升2倍,HotpotQA上准确率提高3.00% EM
  • 适合需要快速响应的开放域问答系统

检索增强生成(RAG)可通过引入外部知识来补充大语言模型(LLM)。然而,随着检索文档数量增加,输入长度线性增长,导致延迟激增且长文本理解能力下降,尤其在需跨文档链式推理的多跳问题中更为严重。为此,本文提出BRIEF(通过证据融合桥接检索与推理),一种轻量级方法:通过查询感知的方式,将检索到的文档压缩为高度密集的文本摘要,融入上下文进行RAG推理。为训练多跳推理的压缩能力,我们利用开源模型从源文档中提取原子命题,构建合成数据以生成合成摘要。基于此合成数据,BRIEF生成更简洁摘要,使多种LLM在开放域问答任务中表现优异。例如,在HotpotQA上,相比当前最优基线,压缩率提升2倍,使用Flan-UL2作为阅读器时,准确率提升3.00% EM和4.16% F1;其摘要比专有GPT-3.5更短,但问答性能几乎相当。

原文摘要 · Abstract (English)

Retrieval-augmented generation (RAG) can supplement large language models (LLMs) by integrating external knowledge. However, as the number of retrieved documents increases, the input length to LLMs grows linearly, causing a dramatic increase in latency and a degradation in long-context understanding. This is particularly serious for multi-hop questions that require a chain of reasoning across documents. To accelerate inference, reduce costs, and minimize distractions, this paper presents BRIEF (Bridging Retrieval and Inference through Evidence Fusion), a lightweight approach that performs query-aware multi-hop reasoning by compressing retrieved documents into highly dense textual summaries to integrate into in-context RAG. To enable learning compression for multi-hop reasoning, we curate synthetic data by extracting atomic propositions that encapsulate distinct factoids from the source documents to compose synthetic summaries. Based on our synthetic data built entirely by open-source models, BRIEF generates more concise summaries and enables a range of LLMs to achieve exceptional open-domain question answering (QA) performance. For example, on HotpotQA, BRIEF improves the compression rate by 2 times compared to the state-of-the-art baseline, while outperforming it by 3.00% EM and 4.16% F1 with Flan-UL2 as the reader model. It also generates more concise summaries than proprietary GPT-3.5, while demonstrating nearly identical QA performance.

多跳推理RAG摘要压缩

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。