用人类偏好优化3D生成，让文字转3D更符合人眼审美。
DreamDPO: Aligning Text-to-3D Generation with Human Preferences via Direct Preference Optimization
- 通过成对比较人类偏好，优化3D内容生成过程。
- 相比现有方法，生成的3D模型质量更高、控制更精细。
- 适合关注3D内容可控生成的研究者与创作者。
文本到3D生成可自动化地从文本描述创建3D内容，在多个领域具有变革潜力。然而，现有方法往往难以与人类偏好对齐，限制了其应用灵活性。为此，本文提出DreamDPO，一种基于直接偏好优化（Direct Preference Optimization）的框架，将人类偏好融入3D生成过程。具体而言，DreamDPO首先构建成对样本，利用奖励模型或大型多模态模型评估其与人类偏好的匹配度，再通过偏好驱动的损失函数优化3D表示。该方法借助成对比较反映偏好，减少对精确点对点质量评估的依赖，同时实现细粒度可控性。实验表明，DreamDPO在生成质量上表现优异，显著优于现有方法，能生成更高品质且更可控的3D内容。代码与模型将开源。
原文摘要 · Abstract (English)
Text-to-3D generation automates 3D content creation from textual descriptions, which offers transformative potential across various fields. However, existing methods often struggle to align generated content with human preferences, limiting their applicability and flexibility. To address these limitations, in this paper, we propose DreamDPO, an optimization-based framework that integrates human preferences into the 3D generation process, through direct preference optimization. Practically, DreamDPO first constructs pairwise examples, then compare their alignment with human preferences using reward or large multimodal models, and lastly optimizes the 3D representation with a preference-driven loss function. By leveraging pairwise comparison to reflect preferences, DreamDPO reduces reliance on precise pointwise quality evaluations while enabling fine-grained controllability through preference-guided optimization. Experiments demonstrate that DreamDPO achieves competitive results, and provides higher-quality and more controllable 3D content compared to existing methods. The code and models will be open-sourced.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。