通过优化初始噪声实现高保真3D图像修复，无需训练。
InpaintSLat: Inpainting Structured 3D Latents via Initial Noise Optimization
- 在3D扩散模型中优化初始噪声，控制生成结构。
- 相比基线方法，上下文一致性和提示对齐显著提升。
- 适合需要精确控制的3D内容编辑与修复场景。
我们提出一种无需训练的可控3D图像修复方法，基于初始噪声优化。在结构化3D隐空间扩散框架中，我们发现几何结构在扩散过程早期即形成，且对初始噪声高度敏感。这一特性导致在修补和编辑任务中难以保持与现有上下文的严格对齐。为此，本文提出在结构化3D隐空间扩散框架中优化初始噪声的策略，确保高保真3D修补。具体而言，利用基于修正流模型的反向传播近似更新初始噪声，并采用专为鲁棒高效3D隐空间优化设计的谱参数化方法。实验表明，该方法在上下文一致性和提示对齐方面持续优于代表性无训练修补基线，确立初始噪声控制作为独立于传统采样轨迹操作的3D修补新维度。
原文摘要 · Abstract (English)
We present a training-free approach for controllable 3D inpainting based on initial noise optimization. In the structured 3D latent diffusion framework, we observe that the underlying geometric structure is established during the early stages of the diffusion process and exhibits high sensitivity to the initial noise. Such characteristics compromise stability in tasks like inpainting and editing, where the model must ensure strict alignment with the existing context while synthesizing a new structure. In this paper, we introduce a strategy to optimize the initial noise within the structured 3D latent diffusion framework, ensuring high-fidelity 3D inpainting. Specifically, we update the initial noise by leveraging a backpropagation approximation grounded in the rectified flow model, with the spectral parameterization specially designed for robust and efficient structured 3D latent optimization. Experiments demonstrate consistent improvements in contextual consistency and prompt alignment over representative training-free inpainting baselines, establishing initial noise control as an independent dimension for 3D inpainting, orthogonal to conventional sampling trajectory manipulation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。