让单图生成的3D场景可自由探索，画质更清晰、视角更连贯。
WonderFree: Enhancing Novel View Quality and Cross-View Consistency for 3D Scene Exploration
- 拆解探索难题为新视角画质与多视角一致性问题，分别用WorldRestorer和ConsistView解决。
- 在多个视角下显著降低浮动物、伪影，用户偏好度提升77.20%。
- 适合需要高沉浸感3D交互的虚拟现实、数字孪生等应用。
从单张图像交互式生成3D场景因其构建沉浸式虚拟世界的能力而受到广泛关注。然而，现有方法在更大范围移动时难以保持高质量渲染，尤其在进入未见区域时表现不佳。为此，我们提出WonderFree，首个支持任意角度与方向自由探索的3D生成模型。将挑战分解为两个关键子问题：新视角画质（消除视觉伪影与漂浮物）和跨视角一致性（保证空间连贯性）。为提升新视角质量，引入WorldRestorer——一个数据驱动的视频修复模型，用于清除浮动物与伪影，并设计数据收集流水线自动获取训练数据，覆盖多样风格场景。此外，提出ConsistView——一种多视角联合修复机制，同步恢复多个视角并保持时空一致性。实验表明，WonderFree不仅在多种视角下提升渲染质量，还显著增强全局连贯性与一致性。CLIP指标与用户研究证实，其相较WonderWorld有77.20%的偏好提升，实现无缝沉浸式3D探索体验。代码、模型与数据将公开。
原文摘要 · Abstract (English)
Interactive 3D scene generation from a single image has gained significant attention due to its potential to create immersive virtual worlds. However, a key challenge in current 3D generation methods is the limited explorability, which cannot render high-quality images during larger maneuvers beyond the original viewpoint, particularly when attempting to move forward into unseen areas. To address this challenge, we propose WonderFree, the first model that enables users to interactively generate 3D worlds with the freedom to explore from arbitrary angles and directions. Specifically, we decouple this challenge into two key subproblems: novel view quality, which addresses visual artifacts and floating issues in novel views, and cross-view consistency, which ensures spatial consistency across different viewpoints. To enhance rendering quality in novel views, we introduce WorldRestorer, a data-driven video restoration model designed to eliminate floaters and artifacts. In addition, a data collection pipeline is presented to automatically gather training data for WorldRestorer, ensuring it can handle scenes with varying styles needed for 3D scene generation. Furthermore, to improve cross-view consistency, we propose ConsistView, a multi-view joint restoration mechanism that simultaneously restores multiple perspectives while maintaining spatiotemporal coherence. Experimental results demonstrate that WonderFree not only enhances rendering quality across diverse viewpoints but also significantly improves global coherence and consistency. These improvements are confirmed by CLIP-based metrics and a user study showing a 77.20% preference for WonderFree over WonderWorld enabling a seamless and immersive 3D exploration experience. The code, model, and data will be publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。