通过频谱演化感知机制，提升扩散模型推理速度与质量。
SeaCache: Spectral-Evolution-Aware Cache for Accelerating Diffusion Models
- 基于频谱对齐表示判断缓存复用，避免内容与噪声混淆。
- 在多个生成模型上实现更快推理且保持高质量输出。
- 无需训练，适配不同内容，适合高效生成场景使用。
扩散模型是视觉生成的强大基础，但其固有的顺序去噪过程导致推理缓慢。以往方法通过基于相邻时间步特征距离的中间输出缓存与重用来加速采样，但现有策略通常依赖原始特征差异，混淆了内容与噪声。该设计忽略了频谱演化规律：低频结构早期出现，高频细节后期细化。本文提出无训练的频谱演化感知缓存（SeaCache），依据频谱对齐表示决定复用策略。理论与实证分析表明，所提出的频谱演化感知（SEA）滤波器能保留内容相关成分并抑制噪声。利用经SEA滤波的输入特征估计冗余性，生成动态缓存调度，既适应内容变化，又尊重扩散模型的频谱先验。在多种视觉生成模型及基线上的大量实验表明，SeaCache实现了当前最优的延迟-质量权衡。
原文摘要 · Abstract (English)
Diffusion models are a strong backbone for visual generation, but their inherently sequential denoising process leads to slow inference. Previous methods accelerate sampling by caching and reusing intermediate outputs based on feature distances between adjacent timesteps. However, existing caching strategies typically rely on raw feature differences that entangle content and noise. This design overlooks spectral evolution, where low-frequency structure appears early and high-frequency detail is refined later. We introduce Spectral-Evolution-Aware Cache (SeaCache), a training-free cache schedule that bases reuse decisions on a spectrally aligned representation. Through theoretical and empirical analysis, we derive a Spectral-Evolution-Aware (SEA) filter that preserves content-relevant components while suppressing noise. Employing SEA-filtered input features to estimate redundancy leads to dynamic schedules that adapt to content while respecting the spectral priors underlying the diffusion model. Extensive experiments on diverse visual generative models and the baselines show that SeaCache achieves state-of-the-art latency-quality trade-offs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。