不训练即可加速扩散模型生成,空间冗余处理提升7倍速度
Just-in-Time: Training-Free Spatial Acceleration for Diffusion Transformers
- 动态选取稀疏锚点令牌,通过近似微分方程驱动全局生成
- 在FLUX.1-dev上实现最高7倍加速,图像质量几乎无损失
- 无需训练、适配主流扩散模型,适合追求高效生成的开发者
扩散变换器在图像生成中达到新顶尖水平,但迭代采样的高计算成本严重阻碍其实际部署。现有加速方法多聚焦时间域,忽略了生成过程中显著的空间冗余——全局结构远早于细节形成。对所有空间区域进行均匀计算是关键低效点。本文提出无需训练的Just-in-Time(JiT)框架,从空间域解决此问题。JiT构建空间近似生成常微分方程(ODE),基于动态选择的稀疏锚点令牌计算,驱动完整潜在状态演化。为确保新增令牌时潜状态维度扩展的平滑过渡,提出确定性微流,一种简单有效的有限时间ODE,保持结构连贯性与统计正确性。在前沿模型FLUX.1-dev上的大量实验表明,JiT实现最高7倍加速,性能近乎无损,显著优于现有方法,建立推理速度与生成保真度之间新的更优权衡。
原文摘要 · Abstract (English)
Diffusion Transformers have established a new state-of-the-art in image synthesis, but the high computational cost of iterative sampling severely hampers their practical deployment. While existing acceleration methods often focus on the temporal domain, they overlook the substantial spatial redundancy inherent in the generative process, where global structures emerge long before fine-grained details are formed. The uniform computational treatment of all spatial regions represents a critical inefficiency. In this paper, we introduce Just-in-Time (JiT), a novel training-free framework that addresses this challenge by acceleration in the spatial domain. JiT formulates a spatially approximated generative ordinary differential equation (ODE) that drives the full latent state evolution based on computations from a dynamically selected, sparse subset of anchor tokens. To ensure seamless transitions as new tokens are incorporated to expand the dimensions of the latent state, we propose a deterministic micro-flow, a simple and effective finite-time ODE that maintains both structural coherence and statistical correctness. Extensive experiments on the state-of-the-art FLUX.1-dev model demonstrate that JiT achieves up to a 7x speedup with nearly lossless performance, significantly outperforming existing acceleration methods and establishing a new and superior trade-off between inference speed and generation fidelity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。