用合成数据优化个性化图像生成，提升概念准确性和提示匹配度。
DreamBoothDPO: Improving Personalized Generation using Direct Preference Optimization
- 用外部质量指标自动生成好坏配对数据，替代人工评分。
- 多步训练使生成质量更高，收敛更快，兼顾概念忠实与提示对齐。
- 可灵活调节图像保真度与文本一致性权衡，适配不同需求。
个性化扩散模型在文生图任务中表现卓越，能将用户定义的概念注入多样场景。然而，如何平衡概念保真度与上下文一致性仍是难题。本文提出一种基于强化学习的方法，利用文生图模型的多样化输出，通过外部质量指标生成合成的成对数据，用于类似直接偏好优化（DPO）的训练。该方法无需人工标注分数，专门设计了更优-更差的配对样本，以同时提升概念保真度和提示遵循性。此外，该方法支持灵活调整图像保真度与文本对齐之间的权衡。多阶段训练实验表明，相比基线方法，本方法在收敛速度和输出质量上均显著提升。我们在多种架构和微调技术下进行了广泛的定性和定量分析，验证了方法的有效性。代码已开源：https://github.com/ControlGenAI/DreamBoothDPO。
原文摘要 · Abstract (English)
Personalized diffusion models have shown remarkable success in Text-to-Image (T2I) generation by enabling the injection of user-defined concepts into diverse contexts. However, balancing concept fidelity with contextual alignment remains a challenging open problem. In this work, we propose an RL-based approach that leverages the diverse outputs of T2I models to address this issue. Our method eliminates the need for human-annotated scores by generating a synthetic paired dataset for DPO-like training using external quality metrics. These better-worse pairs are specifically constructed to improve both concept fidelity and prompt adherence. Moreover, our approach supports flexible adjustment of the trade-off between image fidelity and textual alignment. Through multi-step training, our approach outperforms a naive baseline in convergence speed and output quality. We conduct extensive qualitative and quantitative analysis, demonstrating the effectiveness of our method across various architectures and fine-tuning techniques. The source code can be found at https://github.com/ControlGenAI/DreamBoothDPO.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。