用高效压缩技术把大模型变小,还能保持高质量图像生成能力。
Amber-Image: Efficient Compression of Large-Scale Diffusion Transformers
- 通过分层剪枝和渐进式蒸馏,将60层大模型压缩为10B和6B小模型。
- 参数减少70%,训练仅需不到2000块GPU小时,远低于从零训练。
- 适合资源有限但需高保真图像生成的开发者与部署场景。
扩散变压器(DiT)架构显著推动了文本到图像(T2I)生成的发展,但面临计算成本高昂和部署困难的问题。为此,我们提出一种高效的压缩框架,将基于双流MMDiT的60层Qwen-Image模型转化为无需从头训练的轻量级模型。基于该框架,我们推出了Amber-Image系列轻量化T2I模型。首先,采用时间步敏感的深度剪枝策略构建Amber-Image-10B,保留层通过局部权重平均重初始化,并经逐层蒸馏与全参数微调优化。在此基础上,进一步提出混合流架构,将深层双流结构转为单流,由图像分支初始化,再通过渐进蒸馏与轻量微调优化,得到Amber-Image-6B。该方法使参数量减少70%,并省去大规模数据工程。值得注意的是,从10B到6B模型的整个压缩与训练流程耗时不足2000块GPU小时,远低于从头训练的成本。在DPG-Bench和LongText-Bench等基准测试中,Amber-Image实现高保真合成与优异文本渲染能力,媲美更大规模模型。
原文摘要 · Abstract (English)
Diffusion Transformer (DiT) architectures have significantly advanced Text-to-Image (T2I) generation but suffer from prohibitive computational costs and deployment barriers. To address these challenges, we propose an efficient compression framework that transforms the 60-layer dual-stream MMDiT-based Qwen-Image into lightweight models without training from scratch. Leveraging this framework, we introduce Amber-Image, a series of streamlined T2I models. We first derive Amber-Image-10B using a timestep-sensitive depth pruning strategy, where retained layers are reinitialized via local weight averaging and optimized through layer-wise distillation and full-parameter fine-tuning. Building on this, we develop Amber-Image-6B by introducing a hybrid-stream architecture that converts deep-layer dual streams into a single stream initialized from the image branch, further refined via progressive distillation and lightweight fine-tuning. Our approach reduces parameters by 70% and eliminates the need for large-scale data engineering. Notably, the entire compression and training pipeline-from the 10B to the 6B variant-requires fewer than 2,000 GPU hours, demonstrating exceptional cost-efficiency compared to training from scratch. Extensive evaluations on benchmarks like DPG-Bench and LongText-Bench show that Amber-Image achieves high-fidelity synthesis and superior text rendering, matching much larger models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。