用知识蒸馏和偏好优化，让小模型高效生成高质量图像。
LightGen: Efficient Image Generation through Knowledge Distillation and Direct Preference Optimization
- 用0.7B参数的紧凑模型蒸馏大模型知识，仅需200万合成图像。
- 训练时间从千天级压缩到88天，性能接近顶尖模型。
- 适合算力有限的研究者，提升生成模型可及性。
近期文本到图像生成依赖海量数据和庞大参数模型，严重限制了计算资源有限的研究者使用。本文提出LightGen，一种基于知识蒸馏（KD）与直接偏好优化（DPO）的高效训练范式。受多模态大模型中数据蒸馏成功的启发，LightGen将当前最先进的文本到图像模型的知识，蒸馏至仅含0.7B参数的掩码自回归（MAR）架构中。利用仅200万张由多样化提示生成的高质量合成图像，实验表明数据多样性远超数据量对模型性能的影响。该策略显著降低计算需求，将预训练时间从可能数千个GPU日压缩至仅88个GPU日。此外，为克服合成数据在高频细节与空间准确性上的缺陷，引入DPO技术以提升图像保真度与定位精度。大量实验证实，LightGen在生成质量上可媲美当前最先进模型，同时大幅减少资源消耗，拓展了资源受限环境下的应用可能性。代码已公开于https://github.com/XianfengWu01/LightGen。
原文摘要 · Abstract (English)
Recent advances in text-to-image generation have primarily relied on extensive datasets and parameter-heavy architectures. These requirements severely limit accessibility for researchers and practitioners who lack substantial computational resources. In this paper, we introduce \model, an efficient training paradigm for image generation models that uses knowledge distillation (KD) and Direct Preference Optimization (DPO). Drawing inspiration from the success of data KD techniques widely adopted in Multi-Modal Large Language Models (MLLMs), LightGen distills knowledge from state-of-the-art (SOTA) text-to-image models into a compact Masked Autoregressive (MAR) architecture with only $0.7B$ parameters. Using a compact synthetic dataset of just $2M$ high-quality images generated from varied captions, we demonstrate that data diversity significantly outweighs data volume in determining model performance. This strategy dramatically reduces computational demands and reduces pre-training time from potentially thousands of GPU-days to merely 88 GPU-days. Furthermore, to address the inherent shortcomings of synthetic data, particularly poor high-frequency details and spatial inaccuracies, we integrate the DPO technique that refines image fidelity and positional accuracy. Comprehensive experiments confirm that LightGen achieves image generation quality comparable to SOTA models while significantly reducing computational resources and expanding accessibility for resource-constrained environments. Code is available at https://github.com/XianfengWu01/LightGen
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。