让压缩后的长文档仍能被大模型准确理解，提升检索生成效率。
ArcAligner: Adaptive Recursive Aligner for Compressed Context Embeddings in RAG
- 引入自适应门控机制，仅在复杂信息时才激活额外处理。
- 在多跳和长尾问答任务中，压缩率相近时效果优于现有方法。
- 轻量模块嵌入模型层，不影响推理速度，适合实际部署。
检索增强生成（RAG）有助于大语言模型保持准确性，但将长文档输入提示会导致模型变慢且成本高昂。这促使了上下文压缩技术的发展，包括分词剪枝、摘要生成和基于嵌入的压缩。然而，压缩程度越高，大模型越难理解内容。为此，我们提出ArcAligner（自适应递归上下文对齐器），一个轻量级模块，集成于语言模型层中，帮助模型更好地利用高度压缩的上下文表示进行生成。该模块采用自适应门控系统，仅在信息复杂时启用额外计算，确保系统高效运行。在多个知识密集型问答基准上，ArcAligner在与基线相当的压缩率下持续表现更优，尤其在多跳和长尾场景中优势明显。源代码已公开。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) helps LLMs stay accurate, but feeding long documents into a prompt makes the model slow and expensive. This has motivated context compression, ranging from token pruning and summarization to embedding-based compression. While researchers have tried ''compressing'' these documents into smaller summaries or mathematical embeddings, there is a catch: the more you compress the data, the more the LLM struggles to understand it. To address this challenge, we propose ArcAligner (Adaptive recursive context *Aligner*), a lightweight module integrated into the language model layers to help the model better utilize highly compressed context representations for downstream generation. It uses an adaptive ''gating'' system that only adds extra processing power when the information is complex, keeping the system fast. Across knowledge-intensive QA benchmarks, ArcAligner consistently beats compression baselines at comparable compression rates, especially on multi-hop and long-tail settings. The source code is publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。