提出高效低比特量化框架，让人脸修复模型更轻更快。
QuantFace: Efficient Quantization for Face Restoration
- 通过旋转缩放通道平衡保留激活数据信息
- 6比特和4比特下性能优于现有方法
- 适合部署在资源受限设备的人脸修复场景
扩散模型在人脸修复任务中表现卓越，但计算开销大，限制了实际应用。本文提出QuantFace，一种针对人脸修复模型的新型低比特量化框架，将32位精度的权重和激活量化至4~6比特。我们分析激活数据分布发现其高度变化，为保留原始信息，引入旋转缩放通道平衡机制。进一步提出量化-蒸馏低秩适配（QD-LoRA），联合优化量化与蒸馏效果。最后设计自适应位宽分配策略，将其建模为整数规划问题，综合考虑量化误差与感知指标以实现最优资源分配。在合成与真实世界数据集上的大量实验表明，QuantFace在6比特和4比特下均显著优于当前领先低比特量化方法。代码已开源。
原文摘要 · Abstract (English)
Diffusion models have been achieving remarkable performance in face restoration. However, the heavy computations hamper the widespread adoption of these models. In this work, we propose QuantFace, a novel low-bit quantization framework for face restoration models, where the full-precision (i.e., 32-bit) weights and activations are quantized to 4~6-bit. We first analyze the data distribution within activations and find that it is highly variant. To preserve the original data information, we employ rotation-scaling channel balancing. Furthermore, we propose Quantization-Distillation Low-Rank Adaptation (QD-LoRA), which jointly optimizes for quantization and distillation performance. Finally, we propose an adaptive bit-width allocation strategy. We formulate such a strategy as an integer programming problem that combines quantization error and perceptual metrics to find a satisfactory resource allocation. Extensive experiments on the synthetic and real-world datasets demonstrate the effectiveness of QuantFace under 6-bit and 4-bit. QuantFace achieves significant advantages over recent leading low-bit quantization methods for face restoration. The code is available at https://github.com/jiatongli2024/QuantFace.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。