arXiv:2605.11478cs.AIcs.IT2026-05被引 2

提出新型向量量化方法,显著提升长文本推理中的缓存压缩效率。

FibQuant: Universal Vector Quantization for Random-Access KV-Cache Compression

论文配图:FibQuant: Universal Vector Quantization for Random-Access KV-Cache Compression
图 1 · 摘自论文原文
  • 采用归一化-旋转-分块的通用接口,结合球面贝塔分布建模优化压缩
  • 在GPT-2小模型上实现最高34倍压缩率,保持0.95注意力相似度
  • 支持分数比特压缩,无需校准或可变地址,适合实时随机访问场景

长上下文推理正面临内存流量瓶颈,根源在于键值(KV)缓存随上下文长度、批大小、层数和头数增长,并在每步解码中被读取。基于旋转的标量编码器通过存储范数、应用共享随机旋转、逐坐标量化来满足系统约束,但忽略了归一化带来的几何结构。经哈尔旋转后,一组k个连续坐标构成单位球上的球面-贝塔源。本文提出FibQuant,一种通用固定速率向量量化器,在保持相同归一化-旋转-存储接口的同时,将标量表替换为匹配该典型源的共享径向-角度码本。码本融合贝塔分位数半径、斐波那契/罗伯茨-克罗内克准均匀方向及多重启Lloyd-Max优化。理论证明其向量编码严格优于对应的标量乘积特例,在高率下增益可分解为单元塑形因子与密度匹配因子。相同构造实现密集速率轴,包括分数比特与亚1比特操作点,无需校准或变长地址。在GPT-2 small KV缓存上,FibQuant实现了从5倍压缩率(0.99注意力余弦相似度)到34倍压缩率(0.95)的内存-保真度前沿。端到端测试中,TinyLlama-1.1B在4倍压缩下与fp16仅差0.10困惑度,2比特时(8倍压缩)比标量TurboQuant低3.6倍困惑度,而后者在随机访问量化中已开始失效。

原文摘要 · Abstract (English)

Long-context inference is increasingly a memory-traffic problem. The culprit is the key--value (KV) cache: it grows with context length, batch size, layers, and heads, and it is read at every decoding step. Rotation-based scalar codecs meet this systems constraint by storing a norm, applying a shared random rotation, and quantizing one coordinate at a time. They are universal and random-access, but they discard the geometry created by the normalization step. After a Haar rotation, a block of $k$ consecutive coordinates is not a product source; it is a spherical-Beta source on the unit ball. We introduce \textsc{FibQuant}, a universal fixed-rate vector quantizer that keeps the same normalize--rotate--store interface while replacing scalar tables by a shared radial--angular codebook matched to this canonical source. The codebook combines Beta-quantile radii, Fibonacci\,/\,Roberts--Kronecker quasi-uniform directions, and multi-restart Lloyd--Max refinement. We prove that the resulting vector code strictly improves on its scalar product specialization at matched rate, with a high-rate gain that separates into a cell-shaping factor and a density-matching factor. The same construction gives a dense rate axis, including fractional-bit and sub-one-bit operating points, without calibration or variable-length addresses. On GPT-2 small KV caches, \textsc{FibQuant} traces a memory--fidelity frontier from $5\times$ compression at $0.99$ attention cosine similarity to $34\times$ at $0.95$. End-to-end on TinyLlama-1.1B, it is within $0.10$ perplexity of fp16 at $4\times$ compression and has $3.6\times$ lower perplexity than scalar \textsc{TurboQuant} at $b = 2$ ($8\times$ compression), where scalar random-access quantization begins to fail.

向量量化缓存压缩长文本生成随机访问

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。