arXiv:2602.20497cs.CVcs.AI2026-02中稿 · CVPR被引 3

用可学习的分阶段预测器加速扩散模型,保持生成质量

LESA: Learnable Stage-Aware Predictors for Diffusion Model Acceleration

论文配图:LESA: Learnable Stage-Aware Predictors for Diffusion Model Acceleration
图 1 · 摘自论文原文
  • 设计分阶段多专家网络,用KAN学习不同噪声阶段特征映射
  • 在FLUX.1-dev上提速5倍,视频生成提升24.7%PSNR
  • 适合追求高效高质图像视频生成的研究者与开发者

扩散模型在图像和视频生成中表现卓越,但扩散Transformer(DiT)的高计算需求限制了其实际部署。尽管特征缓存是有效的加速策略,但现有基于简单重用或无训练预测的方法难以适应扩散过程复杂的阶段依赖动态,常导致质量下降且无法保持标准去噪一致性。为此,我们提出一种基于两阶段训练的可学习分阶段预测框架LESA。该方法利用Kolmogorov-Arnold网络(KAN)从数据中精确学习时间特征映射,并引入多阶段多专家架构,为不同噪声水平阶段分配专用预测器,实现更精准稳健的特征预测。大量实验表明,该方法显著加速同时保持高质量生成:在FLUX.1-dev上实现5.00倍加速且质量仅下降1.0%;在Qwen-Image上达6.25倍提速,较此前最优(TaylorSeer)质量提升20.2%;在HunyuanVideo上实现5.00倍加速,PSNR相比TaylorSeer提升24.7%。文本到图像与文本到视频合成均达到顶尖性能,验证了框架在不同模型上的有效性与泛化能力。代码已开源:https://github.com/caipeiliang2004/LESA。

原文摘要 · Abstract (English)

Diffusion models have achieved remarkable success in image and video generation tasks. However, the high computational demands of Diffusion Transformers (DiTs) pose a significant challenge to their practical deployment. While feature caching is a promising acceleration strategy, existing methods based on simple reusing or training-free forecasting struggle to adapt to the complex, stage-dependent dynamics of the diffusion process, often resulting in quality degradation and failing to maintain consistency with the standard denoising process. To address this, we propose a LEarnable Stage-Aware (LESA) predictor framework based on two-stage training. Our approach leverages a Kolmogorov-Arnold Network (KAN) to accurately learn temporal feature mappings from data. We further introduce a multi-stage, multi-expert architecture that assigns specialized predictors to different noise-level stages, enabling more precise and robust feature forecasting. Extensive experiments show our method achieves significant acceleration while maintaining high-fidelity generation. Experiments demonstrate 5.00x acceleration on FLUX.1-dev with minimal quality degradation (1.0% drop), 6.25x speedup on Qwen-Image with a 20.2% quality improvement over the previous SOTA (TaylorSeer), and 5.00x acceleration on HunyuanVideo with a 24.7% PSNR improvement over TaylorSeer. State-of-the-art performance on both text-to-image and text-to-video synthesis validates the effectiveness and generalization capability of our training-based framework across different models. Our code is available at https://github.com/caipeiliang2004/LESA.

扩散模型加速生成KAN

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。