通过分离空间与语义注意力头，实现图像生成的高效压缩。
SSD: Spatial-Semantic Head Decoupling for Efficient Autoregressive Image Generation
- 将注意力头分为空间局部和语义聚焦两类，分别处理缓存。
- 内存降低5倍，吞吐速度提升6.6倍，画质损失极小。
- 适合在低资源设备上部署自回归图像生成模型。
像Janus-Pro这样的自回归图像生成模型虽能生成高质量图像，但因视觉标记数量庞大，导致内存占用高、计算开销持续增长。尽管键值缓存压缩在语言建模中已广泛研究，但在图像生成领域仍基本未被探索。本文首次识别出一种显著的注意力现象：空间局部性与涌现的语义汇聚点。基于此，提出一种新型键值缓存压缩框架：对所有视觉标记的键值缓存进行自适应解耦，空间局部性注意力头仅保留近期标记窗口；语义汇聚型注意力头则战略性保留被高度关注的紧凑标记集合。大量实验表明，该方法实现5倍内存压缩和6.6倍吞吐量提升，且视觉质量损失微乎其微，使资源受限硬件上原生自回归图像生成成为可能。
原文摘要 · Abstract (English)
Autoregressive image generation models like Janus-Pro produce high-quality images, but at the significant cost of high memory and ever-growing computational demands due to the large number of visual tokens. While KV cache compression has been extensively studied in language modeling, it still remains largely unexplored for the image generation domain. In this work, we begin by identifying a distinct and prominent attention phenomenon, which we term spatial locality and emergent semantic sink. To leverage this key insight, we introduce a novel KV cache compression framework. Specifically, we compress the KV cache for all visual tokens by adaptively decoupling attention heads into two separate types: for spatial-locality heads, our method maintains a short recent token window; for semantic-sink heads, it strategically preserves a compact set of highly-attended tokens. Our extensive experiments demonstrate that the proposed method achieves a 5$\times$ reduction in memory usage and a notable 6.6$\times$ speedup in overall throughput with only minimal visual quality loss, thereby enabling highly efficient native autoregressive image generation on resource-constrained hardware.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。