0.72秒生成可交互3D场景，速度提升15倍
WonderTurbo: Generating Interactive 3D World in 0.72 Seconds
- 动态更新几何表示，每步仅需0.26秒
- 实现15倍速度提升，输出高质量且空间一致
- 适合实时虚拟体验与交互式3D应用
交互式3D生成正受到广泛关注，因其能创造沉浸式虚拟体验。然而，现有3D生成技术难以实现实时交互。为此，我们提出WonderTurbo，首个可在0.72秒内生成3D场景新视角的实时交互式3D场景生成框架。该框架加速了3D场景生成中的几何与外观建模。在几何方面，提出StepSplat，一种通过动态更新构建高效3D几何表示的方法，每次更新仅需0.26秒；同时设计轻量级深度补全模块QuickDepth，为StepSplat提供一致深度输入，提升几何精度。在外观建模方面，开发了两步扩散模型FastPaint，专用于即时修复，保持空间外观一致性。实验表明，WonderTurbo相比基线方法实现15倍加速，同时保持优异的空间一致性并输出高质量结果。
原文摘要 · Abstract (English)
Interactive 3D generation is gaining momentum and capturing extensive attention for its potential to create immersive virtual experiences. However, a critical challenge in current 3D generation technologies lies in achieving real-time interactivity. To address this issue, we introduce WonderTurbo, the first real-time interactive 3D scene generation framework capable of generating novel perspectives of 3D scenes within 0.72 seconds. Specifically, WonderTurbo accelerates both geometric and appearance modeling in 3D scene generation. In terms of geometry, we propose StepSplat, an innovative method that constructs efficient 3D geometric representations through dynamic updates, each taking only 0.26 seconds. Additionally, we design QuickDepth, a lightweight depth completion module that provides consistent depth input for StepSplat, further enhancing geometric accuracy. For appearance modeling, we develop FastPaint, a 2-steps diffusion model tailored for instant inpainting, which focuses on maintaining spatial appearance consistency. Experimental results demonstrate that WonderTurbo achieves a remarkable 15X speedup compared to baseline methods, while preserving excellent spatial consistency and delivering high-quality output.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。