arXiv:2512.19206cs.LGcs.AI2025-12ACL被引 5

提出混合精度缓存压缩方法,让长文本推理更省内存且不丢性能。

MixKVQ: Query-Aware Mixed-Precision KV Cache Quantization for Long-Context Reasoning

  • 根据查询动态识别关键通道,只对重要部分保留高精度。
  • 在复杂推理任务上,低比特压缩后性能接近全精度基准。
  • 可直接部署,适合需要长上下文推理的模型优化场景。

长链思维(CoT)推理显著提升了大语言模型的能力,但随之带来的大量键值(KV)缓存导致内存和延迟开销剧增。尽管KV缓存量化是有效的压缩技术,现有低比特量化方法在复杂推理任务中常出现严重性能下降。固定精度量化难以处理键缓存中的异常通道,而现有混合精度策略无法准确识别需高精度表示的组件。我们发现,有效的低比特KV缓存量化需考虑两个因素:键通道的内在量化难度及其与查询的相关性。基于此,我们提出MixKVQ,一种轻量级、查询感知的即插即用方法,能识别并保留关键键通道的高精度表示,同时对值缓存采用逐令牌量化。在复杂推理数据集上的实验表明,该方法显著优于现有低比特方法,在大幅降低内存占用的同时,性能接近全精度基线。

原文摘要 · Abstract (English)

Long Chain-of-Thought (CoT) reasoning has significantly advanced the capabilities of Large Language Models (LLMs), but this progress is accompanied by substantial memory and latency overhead from the extensive Key-Value (KV) cache. Although KV cache quantization is a promising compression technique, existing low-bit quantization methods often exhibit severe performance degradation on complex reasoning tasks. Fixed-precision quantization struggles to handle outlier channels in the key cache, while current mixed-precision strategies fail to accurately identify components requiring high-precision representation. We find that an effective low-bit KV cache quantization strategy must consider two factors: a key channel's intrinsic quantization difficulty and its relevance to the query. Based on this insight, we propose MixKVQ, a novel plug-and-play method that introduces a lightweight, query-aware algorithm to identify and preserve critical key channels that need higher precision, while applying per-token quantization for value cache. Experiments on complex reasoning datasets demonstrate that our approach significantly outperforms existing low-bit methods, achieving performance comparable to a full-precision baseline at a substantially reduced memory footprint.

KV缓存量化长上下文LLM优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。