通过迭代查询增强长文本理解，让大模型更擅长处理复杂长文档。
Enhancing Long Context Performance in LLMs Through Inner Loop Query Mechanism
- 引入迭代查询机制，结合问题与中间结果动态检索
- 在多针堆中找针等长上下文任务上显著提升表现
- 适合需要深度推理的长文档问答场景
Transformer 模型的计算复杂度随输入长度呈二次增长，限制了大语言模型（LLMs）在训练和推理中的上下文窗口大小。尽管基于检索增强生成（RAG）的模型可通过检索系统过滤冗余信息以处理更长上下文，但多数 RAG 方法仅依赖初始查询进行检索，难以应对需深层推理的复杂问题。本文提出一种新方法——内层循环记忆增强树检索（ILM-TR），其在推理时不仅依据原始问题，还结合中间生成结果进行多次迭代查询。模型从 RAG 系统中检索信息，并将来自长文档的多层次抽象数据整合进短期记忆（STM）。基于 STM 内容生成的文本用于构造下一轮查询，该过程持续至 STM 内容收敛。实验表明，该方法在多针堆中找针（M-NIAH）和 BABILong 等长上下文测试中优于传统 RAG 增强模型。
原文摘要 · Abstract (English)
Transformers have a quadratic scaling of computational complexity with input size, which limits the input context window size of large language models (LLMs) in both training and inference. Meanwhile, retrieval-augmented generation (RAG) besed models can better handle longer contexts by using a retrieval system to filter out unnecessary information. However, most RAG methods only perform retrieval based on the initial query, which may not work well with complex questions that require deeper reasoning. We introduce a novel approach, Inner Loop Memory Augmented Tree Retrieval (ILM-TR), involving inner-loop queries, based not only on the query question itself but also on intermediate findings. At inference time, our model retrieves information from the RAG system, integrating data from lengthy documents at various levels of abstraction. Based on the information retrieved, the LLM generates texts stored in an area named Short-Term Memory (STM) which is then used to formulate the next query. This retrieval process is repeated until the text in STM converged. Our experiments demonstrate that retrieval with STM offers improvements over traditional retrieval-augmented LLMs, particularly in long context tests such as Multi-Needle In A Haystack (M-NIAH) and BABILong.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。