用进化算法自动优化扩散模型推理缓存，提速且不降质。
Evolutionary Caching to Accelerate Your Off-the-Shelf Diffusion Model
- 用遗传算法自动生成适合各模型的缓存策略，无需改网络。
- 在PixArt-alpha上提速2.58倍，FID降低4.47点，优于之前最佳方法。
- 策略可跨分辨率和模型变体通用，适合部署时灵活调优。
基于扩散的图像生成模型能生成高质量合成内容，但推理速度慢且计算开销大。以往工作通过在扩散变压器中缓存并重用特征来缓解该问题，但常依赖固定启发式规则，导致加速有限或泛化能力差。本文提出进化缓存加速扩散模型（ECAD），采用遗传算法学习每模型专属的缓存调度，构建帕累托前沿，仅需少量校准提示。ECAD无需修改网络参数或参考图像，可实现显著推理加速，支持精细控制质量与延迟的权衡，并无缝适配不同扩散模型。值得注意的是，其学习的调度能有效泛化至校准阶段未见的分辨率和模型变体。我们在PixArt-alpha、PixArt-Sigma和FLUX.1-dev上使用多种指标（FID、CLIP、Image Reward）和基准（COCO、MJHQ-30k、PartiPrompts）进行评估，结果持续优于先前方法。在PixArt-alpha上，ECAD识别出的调度使性能优于前序最先进方法4.47个COCO FID，同时将推理加速从2.35倍提升至2.58倍。结果表明ECAD是一种可扩展且通用的扩散推理加速方案。
原文摘要 · Abstract (English)
Diffusion-based image generation models excel at producing high-quality synthetic content, but suffer from slow and computationally expensive inference. Prior work has attempted to mitigate this by caching and reusing features within diffusion transformers across inference steps. These methods, however, often rely on rigid heuristics that result in limited acceleration or poor generalization across architectures. We propose Evolutionary Caching to Accelerate Diffusion models (ECAD), a genetic algorithm that learns efficient, per-model, caching schedules forming a Pareto frontier, using only a small set of calibration prompts. ECAD requires no modifications to network parameters or reference images. It offers significant inference speedups, enables fine-grained control over the quality-latency trade-off, and adapts seamlessly to different diffusion models. Notably, ECAD's learned schedules can generalize effectively to resolutions and model variants not seen during calibration. We evaluate ECAD on PixArt-alpha, PixArt-Sigma, and FLUX$.$1-dev using multiple metrics (FID, CLIP, Image Reward) across diverse benchmarks (COCO, MJHQ-30k, PartiPrompts), demonstrating consistent improvements over previous approaches. On PixArt-alpha, ECAD identifies a schedule that outperforms the previous state-of-the-art method by 4.47 COCO FID while increasing inference speedup from 2.35x to 2.58x. Our results establish ECAD as a scalable and generalizable approach for accelerating diffusion inference. Our project page and code are available here: https://research.aniaggarwal.com/ecad
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。