arXiv:2605.15684cs.CV2026-05

ElasticDiT让手机端图像生成在质量与速度间灵活切换,无需多个专用模型。

ElasticDiT: Efficient Diffusion Transformers via Elastic Architecture and Sparse Attention for High-Resolution Image Generation on Mobile Devices

论文配图:ElasticDiT: Efficient Diffusion Transformers via Elastic Architecture and Sparse Attention for High-Resolution Image Generation on Mobile Devices
图 1 · 摘自论文原文
  • 通过动态调整压缩率和网络深度,实现一模型多场景适配。
  • 轻量版在84.16%稀疏度下仍保持32.87的高保真得分,优于Flux模型。
  • 支持即插即用的超轻量VAE和注意力优化,适合移动端部署。

扩散变压器(DiT)是当前高质量图像生成的主流架构,支撑着Stable Diffusion-3和FLUX.1等模型。然而,在资源受限的移动设备上部署这些模型面临巨大的计算与内存开销。尽管线性DiT和静态剪枝等效率方法缓解了瓶颈,但常导致质量下降。与云端不同,移动场景需要单一模型动态平衡保真度与延迟。本文提出ElasticDiT,通过调整空间压缩比和DiT块深度实现动态权衡。结合移位稀疏块注意力(SSBA)与微型DWT蒸馏VAE(T-DVAE),ElasticDiT显著降低推理延迟与内存占用,同时保持图像质量。实验表明,单个ElasticDiT模型可覆盖广泛的保真度-延迟组合。通过联合调节压缩率与深度,其灵活变体在84.16%平均稀疏度下达到32.87的高保真分数(HPS),超越Flux模型;且插件式T-DVAE仅需标准VAE 1/8的计算成本即可实现SD3级重建;Flow-GRPO进一步提升语义对齐,使GenEval从66.93提升至73.62。结果表明,ElasticDiT提供了一种通用、硬件自适应的解决方案,避免了为不同任务设计多个专用模型,为移动端高分辨率图像生成开辟新路径。

原文摘要 · Abstract (English)

The Diffusion Transformer (DiT) architecture is the state-of-the-art paradigm for high-fidelity image generation, underpinning models like Stable Diffusion-3 and FLUX.1. However, deploying these models on resource-constrained mobile devices entails prohibitive computational and memory overhead. While efficiency-driven approaches like Linear-DiT and static pruning alleviate bottlenecks, they often incur quality degradation. Unlike cloud environments, mobile constraints require a single-model paradigm that dynamically balances fidelity and latency. We introduce ElasticDiT, which achieves this dynamic trade-off by adjusting spatial compression ratios and DiT block depths. By integrating Shift Sparse Block Attention (SSBA) and a Tiny DWT-Distilled VAE (T-DVAE), ElasticDiT reduces inference latency and memory footprint while maintaining image quality. Experiments confirm that ElasticDiT effectively covers a wide range of fidelity-latency trade-offs within a single set of parameters. By jointly adjusting compression and depth, a single ElasticDiT model can be reconfigured on-the-fly to outperform task-specific baselines. Specifically, our flex lite variant achieves an HPS of 32.87, surpassing the Flux model, while maintaining competitive quality at 84.16 percent average sparsity through SSBA. Furthermore, the plug-and-play T-DVAE provides SD3-level reconstruction with only 1/8x the computational cost of standard VAEs, and Flow-GRPO boosts semantic alignment (GenEval: 66.93 to 73.62). These results demonstrate that ElasticDiT offers a versatile, hardware-adaptive solution that eliminates the need for multiple specialized models, providing a promising path for future high-resolution image generation on mobile devices.

扩散模型移动端稀疏注意力轻量化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。