针对视频生成模型量化细节丢失问题,提出分阶段对齐的训练与推理方法。
DSAQuant: Denoising-Stage-Aligned Quantization-Aware Training for Video Generation

- 按去噪阶段特性设计量化训练策略,区分不同阶段优化目标。
- 在W3A3下提升VBench评分6.60,显著改善纹理与清晰度。
- 适合追求高效部署且需保持高画质的视频生成应用。
视频扩散模型(VDMs)在文本到视频生成任务中取得显著进展,但其高内存与计算开销限制了实际部署。量化感知训练(QAT)是无需运行时开销即可压缩和加速生成模型的有效方法。然而,现有QAT方法在VDMs中面临独特挑战:虽能保留提示语义、全局布局和粗粒度运动,但量化模型严重损失视觉细节、纹理保真度和锐度。本文追溯此问题根源为传统量化流程的时刻无关设计,忽略了视频去噪的阶段性功能。在VDM中,早期去噪步骤主要建立全局结构与运动,而中后期步骤则细化局部外观与高频细节。基于此洞察,我们提出DSAQuant:一种面向视频生成的去噪阶段对齐量化感知训练框架。训练时,采用去噪阶段导向监督,在早期步骤保留教师蒸馏以稳定结构规划,后期转向目标驱动优化以增强细节重建;推理时,通过去噪阶段门控引导,在最终去噪步骤禁用分类指导(CFG),防止量化引入的误差被放大为高频伪影。在Wan与CogVideoX系列模型上,于W4A4与W3A3设置下的大量实验表明,DSAQuant始终优于当前最优的QAT基线,尤其在激进的W3A3量化下,使VBench平均得分提升达6.60,同时保持强文本-视频对齐性。结果表明,有效的VDM量化不仅需减少量化误差,还需将量化训练与推理与视频扩散的阶段性特性相匹配。
原文摘要 · Abstract (English)
Video diffusion models (VDMs) have achieved impressive progress in text-to-video generation, but their high memory and computational costs hinder practical deployment. Quantization-aware training (QAT) is an effective solution for compressing and accelerating advanced generative models without runtime overhead at inference. However, existing QAT methods suffer from a distinctive challenge in VDMs: while they often preserve prompt semantics, global layout, and coarse motion, the quantized model severely degrades visual details, texture fidelity, and sharpness. In this paper, we trace this degradation to the timestep-agnostic design of conventional quantization pipelines, which overlooks the stage-wise functionality of video denoising. In VDMs, early denoising steps mainly establish global structure and motion, whereas middle and late steps refine local appearance and high-frequency details. Based on this insight, we propose DSAQuant, a Denoising-Stage-Aligned Quantization-aware training framework for VDMs. During training, Denoising-Stage Oriented Supervision preserves teacher distillation in early steps for stable structure planning, while shifting later steps toward target-driven optimization to enhance detail reconstruction. During inference, Denoising-Stage Gated Guidance disables CFG in the final denoising steps to prevent it from amplifying quantization-induced errors into high-frequency artifacts. Extensive experiments on the Wan and CogVideoX families under W4A4 and W3A3 settings show that DSAQuant consistently outperforms the SOTA QAT baseline, improving the VBench average score by up to 6.60 under aggressive W3A3 quantization while preserving strong text-video alignment. These results demonstrate that effective VDM quantization requires not only reducing quantization error, but also aligning quantization training and inference with the stage-wise nature of video diffusion.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。