仅用航拍图生成逼真地面视角视频，实现沉浸式漫游体验。
Skyeyes: Ground Roaming using Aerial View Images
- 融合3D结构与视角一致生成模型，确保图像连贯性。
- 在大视角间隙下仍保持几何一致性与时空连贯性。
- 适用于自动驾驶、游戏等需要真实3D场景的领域。
将基于航拍影像的场景生成应用于自动驾驶和游戏等领域，可提升三维环境的真实感，但对遮挡区域的内容生成及实时一致渲染仍存挑战。本文提出Skyeyes框架，仅利用航拍输入即可生成逼真的地面视角图像序列，实现地面漫游体验。通过结合3D表示与视角一致生成模型，确保生成图像间的一致性，即使存在大视角间隙也能生成几何一致的地面视图。所生成图像具备更优的空间-时间连贯性与真实感，有助于从航拍视角理解与可视化场景。据我们所知，目前尚无公开的地理对齐航拍与地面视角图像数据集，因此我们使用Unreal Engine构建了一个大规模合成的地理对齐数据集。在该合成数据集上的定性和定量分析表明，Skyeyes优于现有主流合成方法。
原文摘要 · Abstract (English)
Integrating aerial imagery-based scene generation into applications like autonomous driving and gaming enhances realism in 3D environments, but challenges remain in creating detailed content for occluded areas and ensuring real-time, consistent rendering. In this paper, we introduce Skyeyes, a novel framework that can generate photorealistic sequences of ground view images using only aerial view inputs, thereby creating a ground roaming experience. More specifically, we combine a 3D representation with a view consistent generation model, which ensures coherence between generated images. This method allows for the creation of geometrically consistent ground view images, even with large view gaps. The images maintain improved spatial-temporal coherence and realism, enhancing scene comprehension and visualization from aerial perspectives. To the best of our knowledge, there are no publicly available datasets that contain pairwise geo-aligned aerial and ground view imagery. Therefore, we build a large, synthetic, and geo-aligned dataset using Unreal Engine. Both qualitative and quantitative analyses on this synthetic dataset display superior results compared to other leading synthesis approaches. See the project page for more results: https://chaoren2357.github.io/website-skyeyes/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。