通过跨尺度不变性缓存，让扩散模型推理快2-3倍且画质几乎无损。
InvarDiff: Cross-Scale Invariance Caching for Accelerated Diffusion Models
- 基于时间步与层间特征不变性，构建可复用模块的二值缓存矩阵。
- 实验显示在DiT和FLUX上实现2-3倍加速，质量指标基本不变。
- 无需训练，适合希望提升生成速度的开发者使用。
扩散模型生成质量高但采样速度慢。我们观察到确定性采样中存在特征不变性，提出InvarDiff——一种无需训练的加速方法，利用时间步尺度与层尺度间的相对时间不变性。通过少量确定性运行，计算每个时间步、每层、每模块的二值缓存计划矩阵，并用重采样校正避免连续缓存导致的漂移。基于分位数变化度量，该矩阵决定哪些模块在哪个步骤可复用而非重新计算。相同不变性准则也应用于步级，决定整个步骤是否可复用缓存结果。推理时，InvarDiff按步骤优先、逐层执行缓存。应用于DiT和FLUX时，该方法减少冗余计算并保持生成质量。实验表明，InvarDiff实现2-3倍端到端加速，对标准质量指标影响极小。定性分析显示，视觉质量几乎无下降。
原文摘要 · Abstract (English)
Diffusion models deliver high-fidelity synthesis but remain slow due to iterative sampling. We empirically observe there exists feature invariance in deterministic sampling, and present InvarDiff, a training-free acceleration method that exploits the relative temporal invariance across timestep-scale and layer-scale. From a few deterministic runs, we compute a per-timestep, per-layer, per-module binary cache plan matrix and use a re-sampling correction to avoid drift when consecutive caches occur. Using quantile-based change metrics, this matrix specifies which module at which step is reused rather than recomputed. The same invariance criterion is applied at the step scale to enable cross-timestep caching, deciding whether an entire step can reuse cached results. During inference, InvarDiff performs step-first and layer-wise caching guided by this matrix. When applied to DiT and FLUX, our approach reduces redundant compute while preserving fidelity. Experiments show that InvarDiff achieves $2$-$3\times$ end-to-end speed-ups with minimal impact on standard quality metrics. Qualitatively, we observe almost no degradation in visual quality compared with full computations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。