不依赖嵌入向量,用大模型自洽推理实现高效长文本问答。
ELITE: Embedding-Less retrieval with Iterative Text Exploration
- 用大模型迭代探索文本,通过新提出的重要度度量优化搜索空间。
- 在NovelQA和Marathon上超越强基线,存储与运行时间降低超十倍。
- 无需图结构或嵌入库,适合资源受限场景下的长文档问答。
大型语言模型在自然语言处理中取得显著进展,但其有限的长期上下文记忆能力制约了文档级或多轮任务的表现。检索增强生成(RAG)通过从外部语料库检索相关信息缓解该问题。然而,现有RAG系统通常依赖基于嵌入的检索,该方法基于语料库层面的语义相似性训练,可能导致检索到的形式语义相似但与问题真实意图不符的内容。此外,近期RAG变体构建图结构或层次结构以提升检索精度,带来显著的计算与存储开销。本文提出一种无嵌入检索框架。该方法利用大模型的逻辑推理能力,在新颖重要度度量引导下进行迭代搜索空间精炼,并在不显式构建图结构的前提下,扩展逻辑相关的信息。在包含NovelQA和Marathon在内的长上下文问答基准上的实验表明,该方法优于强基线,同时存储与运行时间降低超过一个数量级。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have achieved impressive progress in natural language processing, but their limited ability to retain long-term context constrains performance on document-level or multi-turn tasks. Retrieval-Augmented Generation (RAG) mitigates this by retrieving relevant information from an external corpus. However, existing RAG systems often rely on embedding-based retrieval trained on corpus-level semantic similarity, which can lead to retrieving content that is semantically similar in form but misaligned with the question's true intent. Furthermore, recent RAG variants construct graph- or hierarchy-based structures to improve retrieval accuracy, resulting in significant computation and storage overhead. In this paper, we propose an embedding-free retrieval framework. Our method leverages the logical inferencing ability of LLMs in retrieval using iterative search space refinement guided by our novel importance measure and extend our retrieval results with logically related information without explicit graph construction. Experiments on long-context QA benchmarks, including NovelQA and Marathon, show that our approach outperforms strong baselines while reducing storage and runtime by over an order of magnitude.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。