通过推理时搜索优化3D生成质量,无需额外训练。
ITS3D: Inference-Time Scaling for Text-Guided 3D Diffusion Models
- 将噪声搜索建模为优化问题,用验证器反馈迭代改进。
- 在Text-to-3D任务上显著提升生成质量,峰值PSNR提升1.8分。
- 适合追求高质量3D生成且不想重训练的研究者使用。
我们探索了文本引导3D扩散模型中的推理时缩放,以在不增加训练的前提下提升生成质量。为此,提出ITS3D框架,将任务建模为优化问题,寻找最有效的高斯噪声输入。该框架采用验证器引导的搜索算法,根据验证器反馈迭代优化噪声候选。针对3D生成的固有挑战,引入三项技术:1)高斯归一化稳定搜索过程,纠正迭代更新中偏离标准高斯分布的分布偏移;2)基于奇异值分解的压缩技术降低高维3D搜索空间的维度,同时保留有效搜索方向;3)奇异空间重置机制,基于多样性度量动态更新搜索空间,防止收敛至次优局部极小值。大量实验表明,ITS3D显著提升了文本到3D生成质量,展示了计算高效搜索方法在生成过程中的潜力。代码已公开于https://github.com/ZhenglinZhou/ITS3D。
原文摘要 · Abstract (English)
We explore inference-time scaling in text-guided 3D diffusion models to enhance generative quality without additional training. To this end, we introduce ITS3D, a framework that formulates the task as an optimization problem to identify the most effective Gaussian noise input. The framework is driven by a verifier-guided search algorithm, where the search algorithm iteratively refines noise candidates based on verifier feedback. To address the inherent challenges of 3D generation, we introduce three techniques for improved stability, efficiency, and exploration capability. 1) Gaussian normalization is applied to stabilize the search process. It corrects distribution shifts when noise candidates deviate from a standard Gaussian distribution during iterative updates. 2) The high-dimensional nature of the 3D search space increases computational complexity. To mitigate this, a singular value decomposition-based compression technique is employed to reduce dimensionality while preserving effective search directions. 3) To further prevent convergence to suboptimal local minima, a singular space reset mechanism dynamically updates the search space based on diversity measures. Extensive experiments demonstrate that ITS3D enhances text-to-3D generation quality, which shows the potential of computationally efficient search methods in generative processes. The source code is available at https://github.com/ZhenglinZhou/ITS3D.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。