根据查询复杂度动态压缩检索内容，提升大模型问答效率。
Enhancing RAG Efficiency with Adaptive Context Compression
- 按输入复杂度自适应调整压缩率，避免过度或不足压缩。
- 在5个问答数据集上实现4倍以上推理加速，准确率不降反升。
- 适合追求高效精准的AI应用开发与部署人员。
检索增强生成（RAG）通过引入外部知识提升大语言模型性能，但长篇检索上下文带来显著推理开销。现有上下文压缩方法采用固定压缩率，对简单查询过度压缩，对复杂查询则压缩不足。本文提出自适应上下文压缩框架ACC-RAG，根据输入复杂度动态调节压缩率，在不损失准确性的前提下优化推理效率。ACC-RAG结合分层压缩器（用于多粒度嵌入）与上下文选择模块，仅保留最小必要信息，类比人类快速浏览。在Wikipedia及五个QA数据集上的评估显示，相比固定率方法，ACC-RAG表现更优；相较标准RAG，推理速度提升超过4倍，且准确率保持或提高。
原文摘要 · Abstract (English)
Retrieval-augmented generation (RAG) enhances large language models (LLMs) with external knowledge but incurs significant inference costs due to lengthy retrieved contexts. While context compression mitigates this issue, existing methods apply fixed compression rates, over-compressing simple queries or under-compressing complex ones. We propose Adaptive Context Compression for RAG (ACC-RAG), a framework that dynamically adjusts compression rates based on input complexity, optimizing inference efficiency without sacrificing accuracy. ACC-RAG combines a hierarchical compressor (for multi-granular embeddings) with a context selector to retain minimal sufficient information, akin to human skimming. Evaluated on Wikipedia and five QA datasets, ACC-RAG outperforms fixed-rate methods and matches/unlocks over 4 times faster inference versus standard RAG while maintaining or improving accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。