让大模型高效推理外部知识，压缩率30倍且更准
Beyond RAG: Task-Aware KV Cache Compression for Comprehensive Knowledge Reasoning
- 根据任务需求压缩外部知识的键值缓存
- 长文本评测中准确率比RAG高7个百分点
- 适合需要全面知识推理的复杂任务
将外部知识引入大语言模型可提升其在各类应用中的实用性，但现有方法各有不足。检索增强生成（RAG）通过相似性搜索获取证据，但关键信息可能未出现在前几条结果中；长上下文模型虽能处理多文档，却计算成本高且受限于上下文窗口大小。受学生为开卷考试提炼学习材料的启发，我们提出任务感知的键值（KV）缓存压缩方法，在零样本或少样本设置下压缩外部知识，使大模型能高效推理所有相关知识的紧凑表示。实验表明，该方法优于RAG和任务无关压缩方法。在LongBench v2上，相比RAG实现高达7个绝对百分点的准确率提升，压缩率达30倍，推理延迟从0.43秒降至0.16秒。合成数据集显示，当稀疏证据足够时RAG表现良好，而任务感知压缩在广泛知识任务中更具优势。
原文摘要 · Abstract (English)
Incorporating external knowledge in large language models (LLMs) enhances their utility across diverse applications, but existing methods have trade-offs. Retrieval-Augmented Generation (RAG) fetches evidence via similarity search, but key information may fall outside top ranked results. Long-context models can process multiple documents but are computationally expensive and limited by context window size. Inspired by students condensing study material for open-book exams, we propose task-aware key-value (KV) cache compression, which compresses external knowledge in a zero- or few-shot setup. This enables LLMs to reason efficiently over a compacted representation of all relevant information. Experiments show our approach outperforms both RAG and task-agnostic compression methods. On LongBench v2, it improves accuracy by up to 7 absolute points over RAG with a 30x compression rate, while reducing inference latency from 0.43s to 0.16s. A synthetic dataset highlights that RAG performs well when sparse evidence suffices, whereas task-aware compression is superior for broad knowledge tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。