用自监督偏好优化提升文生图模型的全局与细节质量
SUDO: Enhancing Text-to-Image Diffusion Models with Self-Supervised Direct Preference Optimization
- 自监督生成图像偏好对，实现无需标注的数据优化
- 在Stable Diffusion 1.5和XL上显著提升图像整体与局部质量
- 适合希望改进文生图模型但无标注数据的研究者
以往文生图扩散模型多采用监督微调（SFT）来优化预训练模型，但该方法主要最小化像素级均方误差（MSE），忽略了图像层面的全局优化，而后者对感知质量与结构一致性至关重要。本文提出自监督直接偏好优化（SUDO），一种新范式，同时优化像素级细节与图像全局质量。通过将直接偏好优化融入模型，SUDO以自监督方式生成偏好图像对，使模型更关注全局学习，同时补充像素级MSE损失。作为监督微调的有效替代方案，SUDO可无缝应用于任意文生图扩散模型，且无需昂贵的数据收集与标注。在Stable Diffusion 1.5和XL上的大量实验表明，SUDO显著提升了图像的全局与局部质量。代码已开源。
原文摘要 · Abstract (English)
Previous text-to-image diffusion models typically employ supervised fine-tuning (SFT) to enhance pre-trained base models. However, this approach primarily minimizes the loss of mean squared error (MSE) at the pixel level, neglecting the need for global optimization at the image level, which is crucial for achieving high perceptual quality and structural coherence. In this paper, we introduce Self-sUpervised Direct preference Optimization (SUDO), a novel paradigm that optimizes both fine-grained details at the pixel level and global image quality. By integrating direct preference optimization into the model, SUDO generates preference image pairs in a self-supervised manner, enabling the model to prioritize global-level learning while complementing the pixel-level MSE loss. As an effective alternative to supervised fine-tuning, SUDO can be seamlessly applied to any text-to-image diffusion model. Importantly, it eliminates the need for costly data collection and annotation efforts typically associated with traditional direct preference optimization methods. Through extensive experiments on widely-used models, including Stable Diffusion 1.5 and XL, we demonstrate that SUDO significantly enhances both global and local image quality. The codes are provided at \href{https://github.com/SPengLiang/SUDO}{this link}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。