提出修正注意力偏置，让低比特压缩的视频生成更清晰
Quantized Keys Steal Attention: Bias Correction for KV-Cache Compression in Video Diffusion
- 发现量化噪声使缓存键值‘抢夺’注意力，引入动态修正项
- 在INT2下恢复近BF16质量，比INT4还优且内存减半
- 无需额外内存，实时计算，适合长视频生成部署
分块自回归视频扩散模型依赖先前生成块的键值缓存（KV cache）以避免重复计算，但随着视频变长，缓存迅速成为内存瓶颈。现有方法对KV缓存进行低比特量化虽缓解内存压力，却导致视频质量下降。我们发现，质量下降的关键原因是注意力权重中的系统性偏置：由于softmax中指数函数的凸性，量化噪声会放大缓存键值的贡献，这一现象称为詹森偏置（Jensen bias）。该效应使量化键值从当前未量化块窃取注意力。我们推导出一种基于注意力分数的修正方法，在不增加额外内存的前提下，通过缓存键值的量化步长和查询范数实时计算，利用二阶泰勒展开逼近，计算开销可忽略。在MAGI-1、SkyReels-V2和HY-WorldPlay数据集上评估，使用INT2量化时，该方法恢复了大部分因激进量化损失的质量，达到接近BF16的视频质量，并在仅用50%内存的情况下优于INT4量化。
原文摘要 · Abstract (English)
Chunk-wise autoregressive video diffusion models rely on a KV cache of previously generated chunks to avoid redundant computation, but this cache quickly becomes a memory bottleneck as videos grow longer. Methods that quantize the KV cache to low bitwidths reduce memory pressure but degrade video quality. We show that a key driver of this degradation is a systematic bias in attention weights: due to the convexity of the exponential in softmax attention, quantization noise inflates the contribution of cached keys, a phenomenon we call the Jensen bias. This effect causes quantized keys to steal attention mass from the unquantized current chunk. We derive a per-attention-score correction that removes this bias in expectation, computed on the fly from the quantization step sizes of the cached keys and the query norm. Using a second-order Taylor approximation, the additional computational overhead is negligible, and no additional memory is needed alongside the cache. Evaluated on MAGI-1, SkyReels-V2, and HY-WorldPlay at INT2 quantization, our correction recovers most of the quality lost to aggressive quantization, reaching near-BF16 video quality, and can outperform INT4 quantization while using 50% less memory.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。