解决大模型生成中冗余信息与上下文受限问题，提升知识检索效率。
Contextual Compression in Retrieval-Augmented Generation for Large Language Models: A Survey
- 通过压缩检索到的上下文，优化大模型生成时的信息输入。
- 缓解长文本上下文带来的计算开销与无关内容干扰。
- 适合关注大模型知识增强与推理效率的研究者与工程师。
大型语言模型（LLMs）表现出卓越能力，但仍存在幻觉、知识过时、透明度低和推理不可解释等问题。为应对这些挑战，检索增强生成（RAG）被证明是一种可行方案，通过利用外部数据库提升生成内容的一致性与连贯性，尤其在复杂、知识密集型任务中表现突出，并可通过领域特定信息实现持续优化。结合LLM的内在知识与外部数据库的海量动态信息，RAG实现了协同增效。然而，RAG也存在上下文窗口有限、信息冗余以及大量上下文数据带来的高处理开销等局限。本文全面探讨了上下文压缩范式的演进，深入分析该领域现状，最后指出当前挑战并提出潜在研究方向，为该领域的未来发展铺路。
原文摘要 · Abstract (English)
Large Language Models (LLMs) showcase remarkable abilities, yet they struggle with limitations such as hallucinations, outdated knowledge, opacity, and inexplicable reasoning. To address these challenges, Retrieval-Augmented Generation (RAG) has proven to be a viable solution, leveraging external databases to improve the consistency and coherence of generated content, especially valuable for complex, knowledge-rich tasks, and facilitates continuous improvement by leveraging domain-specific insights. By combining the intrinsic knowledge of LLMs with the vast, dynamic repositories of external databases, RAG achieves a synergistic effect. However, RAG is not without its limitations, including a limited context window, irrelevant information, and the high processing overhead for extensive contextual data. In this comprehensive work, we explore the evolution of Contextual Compression paradigms, providing an in-depth examination of the field. Finally, we outline the current challenges and suggest potential research and development directions, paving the way for future advancements in this area.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。