用3D结构补全生成新视角，避免2D修复带来的形状扭曲和视觉错乱。
SceneCompleter: Dense 3D Scene Completion for Generative Novel View Synthesis
- 将新视角生成转化为3D场景稠密补全，联合优化几何与外观
- 在多个数据集上生成的视图在视觉和几何上均保持一致
- 适合需要高保真3D重建与多视角一致性的场景生成任务
生成模型通过强大的图像生成先验，在新视角合成（NVS）中展现出巨大潜力。然而，现有方法通常采用2D修补范式：先补全缺失图像区域，再进行3D重建。这种策略常导致几何失真和外观漂移，因为2D修补模型无法可靠推断跨视角一致生成所需的潜在3D结构。本文提出 extbf{SceneCompleter}，一种几何感知框架，将生成式NVS重新定义为稠密3D场景补全。不同于孤立生成2D视图，SceneCompleter在空间对齐的RGBD隐空间中，通过几何-外观双流扩散模型联合完成几何与外观补全。为提供全局场景上下文，我们进一步引入场景编码器，利用参考图像中的全局语义与风格信息进行条件生成。完成的RGBD预测被对齐并整合到可扩展的3D场景表示中，支持迭代且一致的场景补全。在域内与域外数据集上的大量实验表明，SceneCompleter在多样化场景中均能生成视觉合理且几何一致的新视角。
原文摘要 · Abstract (English)
Generative models have shown great promise for novel view synthesis (NVS) by leveraging strong image generation priors. However, existing approaches typically follow a 2D inpainting paradigm, first completing missing image regions and then performing 3D reconstruction. This strategy often causes geometry distortion and appearance drift, as 2D inpainting models cannot reliably infer the underlying 3D structure required for cross-view consistent generation. In this paper, we propose \textbf{SceneCompleter}, a geometry-aware framework that reformulates generative NVS as dense 3D scene completion. Instead of hallucinating isolated 2D views, SceneCompleter jointly completes geometry and appearance through a geometry-appearance dual-stream diffusion model in a spatially aligned RGBD latent space. To provide holistic scene context, we further introduce a Scene Embedder that conditions generation on global semantic and stylistic information from reference images. The completed RGBD predictions are then aligned and integrated into an expandable 3D scene representation, enabling iterative and coherent scene completion. Extensive experiments on in-domain and out-of-distribution datasets demonstrate that SceneCompleter produces visually plausible and geometrically consistent novel views across diverse scenarios. Project Page: https://chen-wl20.github.io/SceneCompleter
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。