arXiv:2602.13993cs.CV2026-02被引 7

让扩散模型动态跳过冗余计算,速度翻倍且画质几乎不变

Elastic Diffusion Transformer

  • 根据输入内容动态判断每层是否该跳过或缩减计算量
  • 在2D图像和3D资产生成上实现约2倍加速,画质损失极小
  • 适合追求高效推理的AI生成应用开发者

扩散Transformer(DiT)虽有强大生成能力,但计算开销大。现有加速方法依赖固定算力,难以兼顾速度与质量。本文提出弹性扩散Transformer(E-DiT),利用DiT生成过程中的显著稀疏性——不同样本的冗余计算程度差异大。E-DiT为每个DiT模块配备轻量级路由器,根据输入隐变量动态判断该模块是否可跳过;若不跳过,则预测最优MLP宽度压缩比例。推理时引入块级特征缓存机制,基于路由器预测无训练地消除冗余计算。在Qwen-Image、FLUX和Hunyuan3D-3.0等多个2D与3D数据集上验证,实现最高约2倍加速,生成质量几乎无损。代码将开源。

原文摘要 · Abstract (English)

Diffusion Transformers (DiT) have demonstrated remarkable generative capabilities but remain highly computationally expensive. Previous acceleration methods, such as pruning and distillation, typically rely on a fixed computational capacity, leading to insufficient acceleration and degraded generation quality. To address this limitation, we propose \textbf{Elastic Diffusion Transformer (E-DiT)}, an adaptive acceleration framework for DiT that effectively improves efficiency while maintaining generation quality. Specifically, we observe that the generative process of DiT exhibits substantial sparsity (i.e., some computations can be skipped with minimal impact on quality), and this sparsity varies significantly across samples. Motivated by this observation, E-DiT equips each DiT block with a lightweight router that dynamically identifies sample-dependent sparsity from the input latent. Each router adaptively determines whether the corresponding block can be skipped. If the block is not skipped, the router then predicts the optimal MLP width reduction ratio within the block. During inference, we further introduce a block-level feature caching mechanism that leverages router predictions to eliminate redundant computations in a training-free manner. Extensive experiments across 2D image (Qwen-Image and FLUX) and 3D asset (Hunyuan3D-3.0) demonstrate the effectiveness of E-DiT, achieving up to $\sim$2$\times$ speedup with negligible loss in generation quality. Code will be available at https://github.com/wangjiangshan0725/Elastic-DiT.

扩散模型高效推理动态计算图像生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。