解决文本生成3D模型时的几何不一致问题,让不同视角下模型更统一。
RecDreamer: Consistent Text-to-3D Generation via Uniform Score Distillation
- 通过均匀化姿态分布修正先验,消除对标准姿态的偏差。
- 在LLFF、Tanks and Temples等数据集上显著减少重复模式。
- 无需训练的分类器可插拔式提升姿态一致性,适合3D生成研究者。
基于得分蒸馏的文本到3D生成方法常因几何不一致导致3D资产在不同视角下出现重复模式,这一问题被称为多面雅努斯(Multi-Face Janus)问题,源于现有方法难以保持姿态间的一致性且偏向标准姿态。尽管近期工作提升了姿态控制能力,仍受限于固有偏差,影响生成引导。为此,我们提出RecDreamer,通过重塑底层数据分布实现更一致的姿态表示。核心思想是修正先验分布,使姿态变化均匀分布而非偏向标准形式。通过辅助函数修改预设分布,重构密度以满足特定边缘约束,特别确保姿态的边缘分布为均匀分布,从而消除先验知识带来的偏差。我们将此修正后的分布融入现有得分蒸馏算法,称为均匀得分蒸馏。为高效计算辅助函数所需的后验分布,RecDreamer引入一种免训练分类器,以即插即用方式估计姿态类别。同时采用多种噪声状态近似技术,显著提升系统性能。实验结果表明,RecDreamer有效缓解了多面雅努斯问题,在LLFF、Tanks and Temples等数据集上生成的3D资产在不同姿态下具更高一致性。
原文摘要 · Abstract (English)
Current text-to-3D generation methods based on score distillation often suffer from geometric inconsistencies, leading to repeated patterns across different poses of 3D assets. This issue, known as the Multi-Face Janus problem, arises because existing methods struggle to maintain consistency across varying poses and are biased toward a canonical pose. While recent work has improved pose control and approximation, these efforts are still limited by this inherent bias, which skews the guidance during generation. To address this, we propose a solution called RecDreamer, which reshapes the underlying data distribution to achieve a more consistent pose representation. The core idea behind our method is to rectify the prior distribution, ensuring that pose variation is uniformly distributed rather than biased toward a canonical form. By modifying the prescribed distribution through an auxiliary function, we can reconstruct the density of the distribution to ensure compliance with specific marginal constraints. In particular, we ensure that the marginal distribution of poses follows a uniform distribution, thereby eliminating the biases introduced by the prior knowledge. We incorporate this rectified data distribution into existing score distillation algorithms, a process we refer to as uniform score distillation. To efficiently compute the posterior distribution required for the auxiliary function, RecDreamer introduces a training-free classifier that estimates pose categories in a plug-and-play manner. Additionally, we utilize various approximation techniques for noisy states, significantly improving system performance. Our experimental results demonstrate that RecDreamer effectively mitigates the Multi-Face Janus problem, leading to more consistent 3D asset generation across different poses.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。