提出ScaleKV压缩框架,让视觉自回归模型推理内存降为10%。
Memory-Efficient Visual Autoregressive Modeling with Scale-Aware KV Cache Compression
- 按层分组为草稿层与精修层,差异管理缓存需求。
- 在Infinity模型上实现10%内存占用,保持像素级精度。
- 适合需要高效生成图像的部署场景。
视觉自回归(VAR)建模因其创新的逐尺度预测方法,在效率、可扩展性和零样本泛化方面取得显著提升。然而,其自粗到细的机制导致推理过程中KV缓存呈指数增长,引发严重内存消耗和计算冗余。为此,我们提出专为VAR架构设计的ScaleKV缓存压缩框架。该框架基于两个关键观察:不同变压器层的缓存需求差异以及不同尺度下的注意力模式差异。据此,将层划分为草稿层与精修层:草稿层在多尺度间分散注意力,需更大缓存;精修层聚焦当前标记图处理局部细节,缓存需求大幅降低。ScaleKV通过识别各尺度对应的草稿层与精修层,优化多尺度推理流程,实现差异化缓存管理。在先进文本到图像VAR模型家族Infinity上的评估表明,该方法可将所需KV缓存内存降至10%,同时保持像素级保真度。
原文摘要 · Abstract (English)
Visual Autoregressive (VAR) modeling has garnered significant attention for its innovative next-scale prediction approach, which yields substantial improvements in efficiency, scalability, and zero-shot generalization. Nevertheless, the coarse-to-fine methodology inherent in VAR results in exponential growth of the KV cache during inference, causing considerable memory consumption and computational redundancy. To address these bottlenecks, we introduce ScaleKV, a novel KV cache compression framework tailored for VAR architectures. ScaleKV leverages two critical observations: varying cache demands across transformer layers and distinct attention patterns at different scales. Based on these insights, ScaleKV categorizes transformer layers into two functional groups: drafters and refiners. Drafters exhibit dispersed attention across multiple scales, thereby requiring greater cache capacity. Conversely, refiners focus attention on the current token map to process local details, consequently necessitating substantially reduced cache capacity. ScaleKV optimizes the multi-scale inference pipeline by identifying scale-specific drafters and refiners, facilitating differentiated cache management tailored to each scale. Evaluation on the state-of-the-art text-to-image VAR model family, Infinity, demonstrates that our approach effectively reduces the required KV cache memory to 10% while preserving pixel-level fidelity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。