降低音频扩散模型的精度，实现高效部署。
Post-Training Quantization for Audio Diffusion Transformers
- 提出动态量化与平滑优化，减少激活值异常影响。
- 在稳定音频开源模型上实现79%内存减少，保持高音质。
- 适合需要轻量化音频生成系统的开发者使用。
扩散变压器（DiTs）可实现高质量音频合成，但计算开销大、存储需求高，限制了实际应用。本文全面评估了音频DiTs的后训练量化（PTQ）技术，分析静态与动态量化方案的权衡。提出两种实用改进：(1) 基于去噪时间步的自适应平滑方法，按输入通道和时间步调整量化尺度，缓解激活值异常；(2) 基于奇异值分解（SVD）的轻量级低秩适配器（LoRA）分支，补偿残差权重误差。在Stable Audio Open上对W8A8和W4A8配置进行评测，涵盖客观指标与主观听感评分。结果表明，动态量化在低精度下仍能保持音质，静态方法则具有更低延迟。总体而言，低精度DiTs可实现高达79%的内存节省，同时维持高保真生成能力。
原文摘要 · Abstract (English)
Diffusion Transformers (DiTs) enable high-quality audio synthesis but are often computationally intensive and require substantial storage, which limits their practical deployment. In this paper, we present a comprehensive evaluation of post-training quantization (PTQ) techniques for audio DiTs, analyzing the trade-offs between static and dynamic quantization schemes. We explore two practical extensions (1) a denoising-timestep-aware smoothing method that adapts quantization scales per-input-channel and timestep to mitigate activation outliers, and (2) a lightweight low-rank adapter (LoRA)-based branch derived from singular value decomposition (SVD) to compensate for residual weight errors. Using Stable Audio Open we benchmark W8A8 and W4A8 configurations across objective metrics and human perceptual ratings. Our results show that dynamic quantization preserves fidelity even at lower precision, while static methods remain competitive with lower latency. Overall, our findings show that low-precision DiTs can retain high-fidelity generation while reducing memory usage by up to 79%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。