让视频生成模型更轻量，还能保持画质与语义连贯性。
SemanticDialect: Semantic-Aware Mixed-Format Quantization for Video Diffusion Transformers
- 按模块动态选量化格式，像选方言一样适配不同部分
- 在Open-Sora 2.0上接近16位浮点精度，比现有方法好
- 适合做边缘部署的视频生成模型，支持软硬件快速实现
扩散变压器（DiTs）在视频生成任务中表现优异，但其巨大的内存和计算开销限制了在边缘设备上的部署。量化技术可降低资源消耗，但现有方法常因激活值波动大、难以保持语义与时间连贯性而导致画质下降。本文提出SemanticDialect，一种基于语义感知的块级混合格式量化方法。每个模块从预设的格式库（formatbook）中选择最优量化格式（称为“方言”），并借助存储量化误差与索引的查找表，实现低开销的高效格式选择与量化。进一步提出注意力引导的激活分解，通过残差量化减少误差；以及语义感知的方言分配（SeDA），通过强制语义相关令牌采用统一格式，缓解跨标记量化不一致问题。实验表明，SemanticDialect优于现有量化方法及块级格式（MXFP4、NVFP4），在Open-Sora 2.0上接近FP16性能。同时通过RTL设计与GPU核实现验证了其硬件可部署性。
原文摘要 · Abstract (English)
Diffusion Transformers (DiTs) achieve state-of-the-art video generation quality, but their substantial memory and computational footprints hinder edge deployment. Quantization can reduce these costs, yet existing methods often degrade video quality due to high activation variation and the difficulty of preserving semantic and temporal coherence. We propose SemanticDialect, which advances block-wise mixed-format quantization. In this framework, each block selects an optimal format (dialect) from a candidate set (formatbook), which is augmented with lookup tables that store quantization errors and quantized indices, enabling efficient per-block format selection and quantization with minimal online overhead. We further introduce attention-guided activation decomposition, which reduces quantization error via residual quantization, and semantic-aware dialect assignment (SeDA), which reduces cross-token quantization inconsistency by enforcing format uniformity among semantically correlated tokens. Experiments demonstrate that SemanticDialect outperforms prior quantization methods and block-wise formats (MXFP4, NVFP4) while approaching FP16 quality on Open-Sora 2.0. We also validate hardware deployability through RTL design and GPU kernel implementation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。