arXiv:2607.08241cs.CVcs.LG2026-07

针对无分类器扩散模型的量化难题,提出感知引导的混合精度方法

Closing the Null Space: Guidance-Aware Quantization for Classifier-Free Diffusion

  • 基于引导生成结果直接校准,避免条件分支漂移
  • 在20个数据集上实现97.3%的生成质量保留率
  • 适合资源受限下需高保真图像生成的应用场景

在实际计算预算下部署无分类器引导(CFG)扩散模型需进行量化,但现有后训练量化(PTQ)方法将CFG模型视为单分支网络,忽略了其依赖的成对条件/无条件结构。这一结构盲区带来两方面问题:系统层面,双遍推理模式导致延迟开销被参数量和比特操作指标完全掩盖,通用INT8推理无法实现理论效率增益;算法层面,仅以引导差距为校准目标会引入精确零空间——量化模型可完美匹配引导差距诊断,但无条件分支可任意漂移,导致推理时所有引导生成结果被破坏。本文称此为分支漂移陷阱,通过分析证明其存在性,并在实验中发现最佳校准模型的诊断结果与最差样本质量同时出现。为解决该问题,提出感知引导的混合精度(GAMP)方法:直接以引导输出为校准目标,从引导输出退化中推导各层激活位敏感度,并通过贪心背包算法分配比特,从构造上杜绝无条件分支漂移。

原文摘要 · Abstract (English)

Deploying classifier-free guidance (CFG) diffusion models under real-world compute budgets requires quantization, yet existing post-training quantization (PTQ) methods treat CFG models as single-branch networks, ignoring the paired conditional/unconditional structure that CFG inference fundamentally relies on. This structural blind spot has two consequences. At the system level, the two-pass CFG execution pattern imposes a latency overhead that parameter-count and bit-operation metrics conceal entirely, and commodity INT8 inference stacks fail to realize the theoretical efficiency gains that BOPs calculations promise. At the algorithmic level, calibrating against the guidance gap alone admits an exact null space: a quantized model can achieve perfect gap-fidelity diagnostics while the unconditional branch drifts arbitrarily, corrupting every guided prediction at inference time. This paper terms this the branch-drift trap, proves its existence analytically, and confirms it empirically through a false-positive result in which the best-calibrated model by standard diagnostics simultaneously produces the worst sample quality. To close the trap, Guidance-Aware Mixed Precision (GAMP) is proposed, which calibrates directly on the guided prediction, derives per-layer activation-bit sensitivity from guided-output degradation, and allocates bits via a greedy knapsack -- provably preventing unconditional branch drift by construction.

扩散模型量化混合精度

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。