动态调整扩散模型量化精度，减少计算量同时保持质量
TASQ: Temporal-Adaptive Bit Sparsification Quantization for Diffusion Models
- 按时间与空间自适应截断最低有效位，动态降低每阶段精度
- 在多个模型上实现25%-50%的执行周期减少，效果优于静态量化
- 适合追求高效推理的部署场景，尤其对资源受限设备友好
静态量化为每个去噪步骤分配统一权重精度，为保证质量需满足最敏感步骤的精度要求，导致其他可容忍低精度的步骤也承担高开销。为此，本文提出时间自适应位稀疏量化（TASQ），通过共享最高精度权重缓冲区，并学习时空最低有效位掩码，在不复制每阶段权重或运行时搜索的前提下，动态裁剪低位以降低有效精度。存储开销仍由最坏情况决定，但计算量在不敏感阶段显著下降。结合时间精度引擎，该方法将执行周期降低25%至50%，相比静态量化和原始8位串行执行分别提升6.1至7.5倍。代码已开源。
原文摘要 · Abstract (English)
Static quantization assigns one weight precision to every denoising step. To preserve quality, that precision must accommodate the most quantization-sensitive step, even though many other steps can tolerate fewer bits. The resulting model may satisfy its memory budget, but it repeatedly pays worst-case arithmetic throughout the denoising trajectory. We introduce Temporal-Adaptive Bit Sparsification Quantization (TASQ) to separate these two costs. TASQ stores one shared maximum-precision weight buffer and learns a Temporal-Spatial LSB Mask that selects a lower effective precision for each layer and denoising stage by truncating least-significant bits. Storage therefore remains fixed by the worst case, while BitOPs decrease at less sensitive stages without per-stage weight copies or runtime search. A Temporal-Precision Engine maps the learned schedule to bit-serial execution, where cycles scale with effective precision and switching precision has no measured cycle overhead. On PixArt-Sigma, SANA-1.6B, and SDXL-Turbo, TASQ achieves quality comparable to static quantization with less computation. Together with the Temporal-Precision Engine, it reduces execution cycles by 25 to 50 percent over static quantization and by 6.1 to 7.5x over a naive static 8-bit bit-serial execution. Code is available at https://github.com/seokho-han/tasq.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。