针对视频生成模型量化难题，提出新框架Q-VDiT提升压缩后画质与一致性。
Q-VDiT: Towards Accurate Quantization and Distillation of Video-Generation Diffusion Transformers
- 设计分帧感知量化器，补偿令牌与特征维度的量化误差。
- 引入时序保持蒸馏，优化单帧时兼顾整体视频连贯性。
- 在低精度下实现23.40的场景一致性，性能超现有方法1.9倍。
扩散变压器（DiT）在视频生成中表现卓越，但参数量大、计算复杂度高，难以部署于边缘设备。量化可通过降低参数位宽减少存储并加速推理，但现有图像生成模型的量化方法难以适配视频任务。本文识别出两大挑战：量化过程中的信息丢失，以及优化目标与视频生成需求之间的不匹配。为此，提出专为视频DiT设计的量化框架Q-VDiT。从量化角度，提出分帧感知量化估计器（TQE），补偿令牌与特征维度的量化误差；从优化角度，引入时序保持蒸馏（TMD），保留帧间时空相关性，使每帧优化能考虑整体视频上下文。W3A6配置下的Q-VDiT达到23.40的场景一致性，创历史新高，性能优于当前最先进量化方法1.9倍。代码将开源。
原文摘要 · Abstract (English)
Diffusion transformers (DiT) have demonstrated exceptional performance in video generation. However, their large number of parameters and high computational complexity limit their deployment on edge devices. Quantization can reduce storage requirements and accelerate inference by lowering the bit-width of model parameters. Yet, existing quantization methods for image generation models do not generalize well to video generation tasks. We identify two primary challenges: the loss of information during quantization and the misalignment between optimization objectives and the unique requirements of video generation. To address these challenges, we present Q-VDiT, a quantization framework specifically designed for video DiT models. From the quantization perspective, we propose the Token-aware Quantization Estimator (TQE), which compensates for quantization errors in both the token and feature dimensions. From the optimization perspective, we introduce Temporal Maintenance Distillation (TMD), which preserves the spatiotemporal correlations between frames and enables the optimization of each frame with respect to the overall video context. Our W3A6 Q-VDiT achieves a scene consistency of 23.40, setting a new benchmark and outperforming current state-of-the-art quantization methods by 1.9$\times$. Code will be available at https://github.com/cantbebetter2/Q-VDiT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。