轻量级文生图模型SnapGen可在手机上1.4秒生成1024x1024高清图。
SnapGen: Taming High-Resolution Text-to-Image Models for Mobile Devices with Efficient Architectures and Training
- 通过精简架构设计降低参数与延迟，保持生成质量。
- 372M参数模型在ImageNet-1K上达FID 2.06，256x256生成质量顶尖。
- 仅379M参数却超越数十亿参数大模型，适合移动端部署。
现有文生图扩散模型存在模型过大、运行慢、移动端生成质量差等问题。本文提出SnapGen，一个极小且快速的文生图模型，可在移动设备上生成高分辨率高质量图像。首先，系统优化网络架构以减少参数和延迟，同时保证生成质量。其次，采用跨架构知识蒸馏，从更大模型多层级引导小模型从零训练。第三，结合对抗性引导与知识蒸馏，实现少步生成。首次实现在手机端约1.4秒生成1024x1024像素图像。在ImageNet-1K上，仅372M参数的模型生成256x256图像时达到FID 2.06。在T2I基准（GenEval和DPG-Bench）上，仅379M参数的模型超越数十亿参数的大模型，体积比SDXL小7倍，比IF-XL小14倍。
原文摘要 · Abstract (English)
Existing text-to-image (T2I) diffusion models face several limitations, including large model sizes, slow runtime, and low-quality generation on mobile devices. This paper aims to address all of these challenges by developing an extremely small and fast T2I model that generates high-resolution and high-quality images on mobile platforms. We propose several techniques to achieve this goal. First, we systematically examine the design choices of the network architecture to reduce model parameters and latency, while ensuring high-quality generation. Second, to further improve generation quality, we employ cross-architecture knowledge distillation from a much larger model, using a multi-level approach to guide the training of our model from scratch. Third, we enable a few-step generation by integrating adversarial guidance with knowledge distillation. For the first time, our model SnapGen, demonstrates the generation of 1024x1024 px images on a mobile device around 1.4 seconds. On ImageNet-1K, our model, with only 372M parameters, achieves an FID of 2.06 for 256x256 px generation. On T2I benchmarks (i.e., GenEval and DPG-Bench), our model with merely 379M parameters, surpasses large-scale models with billions of parameters at a significantly smaller size (e.g., 7x smaller than SDXL, 14x smaller than IF-XL).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。