用统一3D空间生成场景，让多视角更一致
OneWorld: Taming Scene Generation with 3D Unified Representation Autoencoder
- 在统一3D隐空间中直接做扩散生成
- 跨视角一致性显著优于现有2D方法
- 适合做高质量3D场景生成的研究者
现有基于扩散的3D场景生成方法主要在2D图像/视频隐空间中操作，难以保证多视角外观与几何一致性。为此，我们提出OneWorld框架，直接在统一的3D表示空间中进行扩散生成。核心是3D统一表示自编码器（3D-URAE），它利用预训练3D基础模型，并通过注入外观信息和蒸馏语义，将其几何主导特性扩展到统一3D隐空间。此外，我们引入词元级跨视图对应（CVC）一致性损失，显式强制多视图结构对齐；提出流形漂移强迫（MDF）机制，通过混合漂移与原始表示，缓解训练-推理暴露偏差，构建稳健的3D流形。大量实验表明，OneWorld生成的3D场景质量高，且跨视角一致性显著优于当前最先进的2D方法。代码将开源于https://github.com/SensenGao/OneWorld。
原文摘要 · Abstract (English)
Existing diffusion-based 3D scene generation methods primarily operate in 2D image/video latent spaces, which makes maintaining cross-view appearance and geometric consistency inherently challenging. To bridge this gap, we present OneWorld, a framework that performs diffusion directly within a coherent 3D representation space. Central to our approach is the 3D Unified Representation Autoencoder (3D-URAE); it leverages pretrained 3D foundation models and augments their geometry-centric nature by injecting appearance and distilling semantics into a unified 3D latent space. Furthermore, we introduce token-level Cross-View-Correspondence (CVC) consistency loss to explicitly enforce structural alignment across views, and propose Manifold-Drift Forcing (MDF) to mitigate train-inference exposure bias and shape a robust 3D manifold by mixing drifted and original representations. Comprehensive experiments demonstrate that OneWorld generates high-quality 3D scenes with superior cross-view consistency compared to state-of-the-art 2D-based methods. Our code will be available at https://github.com/SensenGao/OneWorld.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。