用卫星图生成逼真地面视角,解决超远俯视图重建难题
From Orbit to Ground: Generative City Photogrammetry from Extreme Off-Nadir Satellite Images
- 用2.5D高程图建模城市,稳定稀疏卫星图像下的几何优化
- 通过可微渲染和生成修复网络,还原模糊卫星图中的高清纹理
- 在4平方公里区域仅用几张卫星图实现顶尖视觉效果,适合城市规划
从极低分辨率的卫星图像进行城市级三维重建面临极端视角外推问题,目标是从少量轨道影像中合成地面视角的新视图,但这些图像存在严重畸变和纹理缺失,导致现有方法如NeRF和3DGS失效。为此,本文提出两种针对城市结构与卫星输入特点的设计:首先,将城市几何建模为2.5D高度图,采用Z单调符号距离场(SDF),匹配自上而下的建筑布局,提升在稀疏、大倾角卫星视图下的优化稳定性,生成具有清晰屋顶和垂直拉伸立面的封闭网格;其次,通过可微渲染技术从卫星图像中绘制网格外观。尽管输入图像存在远距离模糊,仍进一步训练生成式纹理修复网络,从退化输入中恢复高频且合理的纹理细节。大量实验验证了方法的可扩展性与鲁棒性,例如在示例中仅用几张卫星图即完成4 km²真实区域重建,合成的地面视图达到当前最优水平。生成模型不仅视觉逼真,还可作为高保真、可直接应用的城市资产,服务于城市规划与仿真等下游任务。
原文摘要 · Abstract (English)
City-scale 3D reconstruction from satellite imagery presents the challenge of extreme viewpoint extrapolation, where our goal is to synthesize ground-level novel views from sparse orbital images with minimal parallax. This requires inferring nearly $90^\circ$ viewpoint gaps from image sources with severely foreshortened facades and flawed textures, causing state-of-the-art reconstruction engines such as NeRF and 3DGS to fail. To address this problem, we propose two design choices tailored for city structures and satellite inputs. First, we model city geometry as a 2.5D height map, implemented as a Z-monotonic signed distance field (SDF) that matches urban building layouts from top-down viewpoints. This stabilizes geometry optimization under sparse, off-nadir satellite views and yields a watertight mesh with crisp roofs and clean, vertically extruded facades. Second, we paint the mesh appearance from satellite images via differentiable rendering techniques. While the satellite inputs may contain long-range, blurry captures, we further train a generative texture restoration network to enhance the appearance, recovering high-frequency, plausible texture details from degraded inputs. Our method's scalability and robustness are demonstrated through extensive experiments on large-scale urban reconstruction. For example, in our teaser figure, we reconstruct a $4\,\mathrm{km}^2$ real-world region from only a few satellite images, achieving state-of-the-art performance in synthesizing photorealistic ground views. The resulting models are not only visually compelling but also serve as high-fidelity, application-ready assets for downstream tasks like urban planning and simulation. Project page can be found at https://pku-vcl-geometry.github.io/Orbit2Ground/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。