arXiv:2506.00600cs.CV2025-06被引 1

从卫星图生成连贯的360度地面视角场景,无需额外辅助数据。

SatDreamer360: Multiview-Consistent Generation of Ground-Level Scenes from Satellite Imagery

  • 用三平面表示法和射线注意力机制,从卫星图提取视点相关特征。
  • 在已知路径下生成多视角一致的全景图,大幅提升场景连贯性。
  • 适合自动驾驶仿真与数字孪生城市构建,尤其看重视角一致性。

从卫星影像生成多视角一致的360°地面场景是一项具有广泛应用价值的挑战性任务,涵盖模拟、自动驾驶导航和数字孪生城市等。现有方法多聚焦于生成单个地面全景图,常依赖高程图或手工投影等辅助输入,难以保证多视角一致性。本文提出SatDreamer360框架,仅需一张卫星图像和预定义的相机姿态轨迹,即可生成几何一致的多视角地面全景图。为解决地面与卫星视角的巨大差异,采用三平面表示编码场景特征,并设计基于射线的像素注意力机制,从三平面中检索视点特定特征。为保持多帧一致性,引入全景对极约束注意力模块,基于已知相对位姿对齐跨帧特征。为支持评估,我们扩充原始VIGOR数据集,构建了大型数据集VIGOR++,包含更多地面视角图像及其姿态标注。实验表明,SatDreamer360在卫星到地面的对齐精度和多视角一致性方面均优于现有方法。

原文摘要 · Abstract (English)

Generating multiview-consistent $360^\circ$ ground-level scenes from satellite imagery is a challenging task with broad applications in simulation, autonomous navigation, and digital twin cities. Existing approaches primarily focus on synthesizing individual ground-view panoramas, often relying on auxiliary inputs like height maps or handcrafted projections, and struggle to produce multiview consistent sequences. In this paper, we propose SatDreamer360, a framework that generates geometrically consistent multi-view ground-level panoramas from a single satellite image, given a predefined pose trajectory. To address the large viewpoint discrepancy between ground and satellite images, we adopt a triplane representation to encode scene features and design a ray-based pixel attention mechanism that retrieves view-specific features from the triplane. To maintain multi-frame consistency, we introduce a panoramic epipolar-constrained attention module that aligns features across frames based on known relative poses. To support the evaluation, we introduce {VIGOR++}, a large-scale dataset for generating multi-view ground panoramas from a satellite image, by augmenting the original VIGOR dataset with more ground-view images and their pose annotations. Experiments show that SatDreamer360 outperforms existing methods in both satellite-to-ground alignment and multiview consistency.

360度生成卫星影像多视角一致数字孪生

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。