让扩散Transformer在手机上高效生成高清图像。
SnapGen++: Unleashing Diffusion Transformers for Efficient High-Fidelity Image Generation on Edge Devices
- 设计轻量级架构，用稀疏注意力兼顾全局与细节。
- 支持4步生成，延迟低，适合实时移动端使用。
- 单模型适配不同设备，部署灵活且质量高。
扩散Transformer（DiT）在图像生成领域达到新高度，但因计算与内存开销大，难以在移动和边缘设备上部署。本文提出SnapGen++，一种面向终端设备的高效DiT框架，在严格资源限制下实现变压器级别生成质量。设计包含三部分：首先，提出紧凑型DiT架构，采用自适应全局-局部稀疏注意力机制，在建模全局上下文与保持局部细节间取得平衡；其次，提出弹性训练框架，联合优化统一超网络中不同容量的子模型，使单一模型可动态适配不同硬件进行高效推理；最后，开发知识引导分布匹配蒸馏（Knowledge-Guided Distribution Matching Distillation），通过步骤蒸馏集成DMD目标与少步教师模型的知识，生成高质量、低延迟结果（如4步生成），适用于实时终端应用。上述贡献共同实现了多样硬件上的可扩展、高效且高质量的扩散模型部署。
原文摘要 · Abstract (English)
Recent advances in diffusion transformers (DiTs) have set new standards in image generation, yet remain impractical for on-device deployment due to their high computational and memory costs. In this work, we present an efficient DiT framework tailored for mobile and edge devices that achieves transformer-level generation quality under strict resource constraints. Our design combines three key components. First, we propose a compact DiT architecture with an adaptive global-local sparse attention mechanism that balances global context modeling and local detail preservation. Second, we propose an elastic training framework that jointly optimizes sub-DiTs of varying capacities within a unified supernetwork, allowing a single model to dynamically adjust for efficient inference across different hardware. Finally, we develop Knowledge-Guided Distribution Matching Distillation, a step-distillation pipeline that integrates the DMD objective with knowledge transfer from few-step teacher models, producing high-fidelity and low-latency generation (e.g., 4-step) suitable for real-time on-device use. Together, these contributions enable scalable, efficient, and high-quality diffusion models for deployment on diverse hardware.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。