arXiv:2512.12604cs.CV2025-12被引 2

通过分层缓存策略,大幅加速扩散模型生成而不明显损失质量。

No Cache Left Idle: Accelerating diffusion model via Extreme-slimming Caching

  • 设计双阈值控制器,分阶段推进缓存复用,先大范围提速再精细修复。
  • 在FLUX.1-dev上实现4.97倍加速,对齐真实感知质量,误差可控。
  • 无需训练,适配多类扩散模型,适合追求高效推理的开发者。

扩散模型生成质量卓越,但计算开销随步数、模型深度和序列长度增加而上升。特征缓存有效,因相邻时间步特征高度相似。然而,存在固有权衡:激进的时间步复用虽能大幅提升速度,却易突破保真度临界点;块或标记级复用更安全,但加速效果有限。我们提出X-Slim(eXtreme-Slimming Caching),据我们所知,首个统一利用时间步、结构(块)与空间(标记)层面可缓存冗余的免训练缓存加速框架。不同于简单混合层级,X-Slim引入双阈值控制器,将缓存过程转化为‘推-修’模式:先将时间步级复用推至预警线,再切换至轻量块与标记级刷新以修补残余冗余,一旦接近临界线则触发完整推理重置累积误差。各层级均采用上下文感知指标决定缓存时机与位置。在多种任务中,X-Slim显著提升速度-质量平衡。在FLUX.1-dev和HunyuanVideo上,延迟分别降低最多4.97倍和3.52倍,感知损失极小。在DiT-XL/2上,加速达3.13倍,且FID较此前方法改善2.42。

原文摘要 · Abstract (English)

Diffusion models achieve remarkable generative quality, but computational overhead scales with step count, model depth, and sequence length. Feature caching is effective since adjacent timesteps yield highly similar features. However, an inherent trade-off remains: aggressive timestep reuse offers large speedups but can easily cross the critical line, hurting fidelity, while block- or token-level reuse is safer but yields limited computational savings. We present X-Slim (eXtreme-Slimming Caching), a training-free, cache-based accelerator that, to our knowledge, is the first unified framework to exploit cacheable redundancy across timesteps, structure (blocks), and space (tokens). Rather than simply mixing levels, X-Slim introduces a dual-threshold controller that turns caching into a push-then-polish process: it first pushes reuse at the timestep level up to an early-warning line, then switches to lightweight block- and token-level refresh to polish the remaining redundancy, and triggers full inference once the critical line is crossed to reset accumulated error. At each level, context-aware indicators decide when and where to cache. Across diverse tasks, X-Slim advances the speed-quality frontier. On FLUX.1-dev and HunyuanVideo, it reduces latency by up to 4.97x and 3.52x with minimal perceptual loss. On DiT-XL/2, it reaches 3.13x acceleration and improves FID by 2.42 over prior methods.

扩散模型缓存加速推理优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。