LoRaQ让4比特量化模型性能更优，无需高精度分支和校准。
LoRaQ: Optimized Low Rank Approximation for 4-bit Quantization
- 用低秩补偿替代高精度分支，实现全4比特流水线
- 在相同内存开销下，超越当前最优方法的生成效果
- 支持混合精度配置，适合现代量化硬件部署
后训练量化（PTQ）对在资源受限设备上部署大型扩散变换器至关重要，但激进的4比特量化会显著降低生成性能。低秩近似方法通过添加辅助线性分支来恢复性能，成为有前景的解决方案。然而，现有最优方法假设这些分支必须保持高精度（W16A16），且依赖复杂的、数据相关的校准进行初始化。我们提出LoRaQ（低秩近似量化），一种简单、无数据校准的方法，优化量化误差补偿。通过消除对高精度分支的需求，LoRaQ首次实现完全低于16比特的流水线，使低秩分支自身也可被量化。我们在Pixart-Σ和SANA上验证，在相同内存开销下，LoRaQ优于现有方法的原生实现。我们还分析了混合精度配置，表明低秩分支采用W8A8、W6A6或W4A8，主层为W4时，可获得更优结果，同时保持与现代混合精度硬件兼容的全量化架构。
原文摘要 · Abstract (English)
Post-training quantization (PTQ) is essential for deploying large diffusion transformers on resource-constrained hardware, but aggressive 4-bit quantization significantly degrades generative performance. Low-rank approximation methods have emerged as a promising solution by appending auxiliary linear branches to restore performance. However, current state-of-the-art approaches assume these branches must retain high precision (W16A16) and rely on heavy, data-dependent calibration for initialization. We challenge both limitations with LoRaQ (Low-Rank Approximated Quantization), a simple, data-free calibration approach that optimizes quantization error compensation. By overcoming the need for high-precision branches, LoRaQ enables the first fully sub-16 bit pipeline, allowing the low-rank branch itself to be quantized. We demonstrate that, at equal memory overhead, LoRaQ outperforms the state-of-the-art methods in their native implementations on Pixart-$Σ$ and SANA. We also analyze mixed-precision configurations, showing that setups such as W8A8, W6A6, and W4A8 for the low-rank branch, alongside a W4 main layer, yield superior results while maintaining a fully quantized architecture compatible with modern mixed-precision hardware.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。