用卫星图生成可沉浸探索的3D城市，无需昂贵标注
Skyfall-GS: Synthesizing Immersive 3D Urban Scenes from Satellite Imagery
- 结合卫星图重建粗略结构，用扩散模型细化细节
- 生成城市街区级场景，视角间几何一致且纹理逼真
- 适合虚拟现实、自动驾驶等需要真实3D环境的场景
生成大规模、可探索且几何准确的3D城市场景，对沉浸式和具身应用具有重要意义。挑战在于缺乏大规模高质量的真实世界3D扫描数据以训练通用生成模型。本文提出一种新方法：利用现成的卫星影像生成逼真的粗略几何结构，并结合开放域扩散模型实现高保真近景外观合成。我们提出Skyfall-GS，一个混合框架，通过融合卫星重建与扩散模型精炼，生成街区尺度的沉浸式3D城市场景，无需昂贵的3D标注，支持实时沉浸式探索。设计了一种课程驱动的迭代优化策略，逐步提升几何完整性和照片级纹理质量。大量实验表明，相比现有最优方法，Skyfall-GS在跨视角几何一致性与纹理真实感方面均有显著提升。
原文摘要 · Abstract (English)
Synthesizing large-scale, explorable, and geometrically accurate 3D urban scenes is a challenging yet valuable task for immersive and embodied applications. The challenge lies in the lack of large-scale and high-quality real-world 3D scans for training generalizable generative models. In this paper, we take an alternative route to create large-scale 3D scenes by leveraging readily available satellite imagery for realistic coarse geometry and open-domain diffusion models for high-quality close-up appearance synthesis. We propose Skyfall-GS, a novel hybrid framework that synthesizes immersive city-block scale 3D urban scenes by combining satellite reconstruction with diffusion refinement, eliminating the need for costly 3D annotations, and also featuring real-time, immersive 3D exploration. We tailor a curriculum-driven iterative refinement strategy to progressively enhance geometric completeness and photorealistic texture. Extensive experiments demonstrate that Skyfall-GS provides improved cross-view consistent geometry and more realistic textures compared to state-of-the-art approaches. Project page: https://skyfall-gs.jayinnn.dev/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。