轻量化视觉自回归模型，三方面优化压缩，高效生成图像。
LiteVAR: Compressing Visual Autoregressive Modelling with Efficient Attention and Quantization
- 设计高效注意力机制与低比特量化，减少冗余计算和存储。
- 注意力计算减少85.2%，内存降低50%，延迟提升1.5倍，性能损失<0.056 FID。
- 无需训练的压缩方法，适合边缘设备部署，兼顾效率与可行性。
视觉自回归（VAR）在图像生成中展现出与基于扩散模型相当的潜力，但现有模型计算开销大，难以在资源受限设备上应用。本文分析发现VAR模型存在三方面冗余：（1）注意力图，（2）使用无分类器引导时的注意力输出，（3）数据精度。针对此，提出高效注意力机制与低比特量化方法，在性能损失低于0.056 FID的前提下，实现85.2%的注意力计算缩减、50%的内存降低及1.5倍的延迟优化。同时开发了无需训练的压缩技术，评估各方法在部署中的可行性与效率增益。
原文摘要 · Abstract (English)
Visual Autoregressive (VAR) has emerged as a promising approach in image generation, offering competitive potential and performance comparable to diffusion-based models. However, current AR-based visual generation models require substantial computational resources, limiting their applicability on resource-constrained devices. To address this issue, we conducted analysis and identified significant redundancy in three dimensions of the VAR model: (1) the attention map, (2) the attention outputs when using classifier free guidance, and (3) the data precision. Correspondingly, we proposed efficient attention mechanism and low-bit quantization method to enhance the efficiency of VAR models while maintaining performance. With negligible performance lost (less than 0.056 FID increase), we could achieve 85.2% reduction in attention computation, 50% reduction in overall memory and 1.5x latency reduction. To ensure deployment feasibility, we developed efficient training-free compression techniques and analyze the deployment feasibility and efficiency gain of each technique.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。