提出无需数据的量化方法，让扩散模型推理更快更省资源。
OrbitQuant: Data-Agnostic Quantization for Image and Video Diffusion Transformers
- 用随机旋转基代替传统范围估计，统一处理不同时间步和提示。
- 在图像与视频模型上实现低至W2A4的量化，仍保持可用生成质量。
- 无需针对新模型或模态重新校准，适合快速部署与跨领域应用。
扩散变换器（DiTs）在图像与视频生成中表现卓越，但其多步采样和参数量增长导致推理成本高昂。后训练量化（PTQ）是自然解决方案，然而DiT激活值随时间步、提示词和引导分支变化，迫使已有方法为每个新检查点或模态重新校准数据。本文提出OrbitQuant，一种数据无关的权值-激活量化方法，通过在归一化旋转基中量化，避免范围估计。该基下，随机置换块哈达玛（RPBH）旋转使各坐标集中于固定已知边缘分布，因此单个Lloyd-Max码本可覆盖所有时间步、提示与层。我们将同一量化器扩展至离线权重行量化，将旋转嵌入权重，使旋转在每层内抵消，仅需运行时对激活进行前向旋转。该方案从图像无缝迁移至视频，无需调参。在FLUX.1、Z-Image-Turbo、Wan 2.1和CogVideoX上，该方法在多个低比特设置下达到当前PTQ最优表现，同时将图像扩散模型量化推进至W2A4且保持可用生成质量。
原文摘要 · Abstract (English)
Diffusion transformers (DiTs) achieve state-of-the-art image and video generation, but their multi-step sampling and growing parameter count make inference expensive. Post-training quantization (PTQ) is the natural remedy, yet DiT activations shift across timesteps, prompts, and guidance branches, forcing prior methods to re-fit calibration data for every new checkpoint or modality. We present OrbitQuant, a data-agnostic weight-activation quantizer that bypasses range estimation by quantizing in a normalized, rotated basis. In this basis, a randomized permuted block-Hadamard (RPBH) rotation concentrates each coordinate around one fixed, known marginal regardless of the input, so a single Lloyd-Max codebook serves all timesteps, prompts, and layers of a given input dimension. We extend the same quantizer to weight rows offline, absorbing the rotation into the weights so that it cancels inside each linear layer and only a forward rotation on the activations remains at runtime. The same recipe transfers from image to video with no per-modality tuning. Across FLUX.1, Z-Image-Turbo, Wan 2.1, and CogVideoX, it sets the state of the art for PTQ at several low-bit settings. It also pushes PTQ of image diffusion transformers to W2A4 with usable generation quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。