将卫星图转为逼真街景全景，支持多视角一致渲染
Seeing through Satellite Images at Street Views
- 通过神经辐射场建模街景特有元素（如天空、光照）
- 在城郊数据集上实现与卫星图一致的逼真街景生成
- 适合城市规划、自动驾驶等需要街景重建的场景
本文研究卫星图到街景图像的合成任务，旨在根据任意卫星图像及指定相机位置或轨迹，生成逼真的街景全景图像与视频。我们提出从卫星与街景视角配对图像中学习神经辐射场，由于视角跨度极大且街景视角稀疏，该问题极具挑战性。基于街景特有元素（如天空、光照）仅在街景全景中可见的观察，提出新方法 Sat2Density++，通过在神经网络中建模这些元素，实现高质量街景全景渲染。实验表明，该方法在城郊场景数据集上能生成多视角一致、忠实于卫星图的逼真街景图像。
原文摘要 · Abstract (English)
This paper studies the task of SatStreet-view synthesis, which aims to render photorealistic street-view panorama images and videos given any satellite image and specified camera positions or trajectories. We formulate to learn neural radiance field from paired images captured from satellite and street viewpoints, which comes to be a challenging learning problem due to the sparse-view natural and the extremely-large viewpoint changes between satellite and street-view images. We tackle the challenges based on a task-specific observation that street-view specific elements, including the sky and illumination effects are only visible in street-view panoramas, and present a novel approach Sat2Density++ to accomplish the goal of photo-realistic street-view panoramas rendering by modeling these street-view specific in neural networks. In the experiments, our method is testified on both urban and suburban scene datasets, demonstrating that Sat2Density++ is capable of rendering photorealistic street-view panoramas that are consistent across multiple views and faithful to the satellite image.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。