通过迭代校正姿态,实现卫星图到街景图的精准生成与环境控制。
Controllable Satellite-to-Street-View Synthesis with Precise Pose Alignment and Zero-Shot Environmental Control
- 在去噪过程中引入迭代单应性调整,解决生成图像姿态错位问题。
- 生成图像姿态误差降低42%,且在多种光照天气下保持空间一致性。
- 支持零样本环境控制,适合城市规划、自动驾驶等场景应用。
从卫星图像生成街景图是一项挑战性任务,尤其在保持精确姿态对齐和融入多样环境条件方面。尽管扩散模型在生成任务中表现出色,但其在扩散过程中难以维持严格的姿态对齐。本文提出一种新颖的迭代单应性调整(IHA)方案,应用于去噪过程,有效解决姿态错位问题,确保生成街景图像的空间一致性。此外,现有卫星到街景生成数据集在光照与天气多样性上有限,限制了生成结果的泛化能力。为此,我们引入文本引导的光照与天气控制采样策略,实现对环境因素的细粒度调控。大量定量与定性评估表明,该方法显著提升姿态准确性,并增强生成图像的多样性与真实感,为卫星到街景生成任务树立了新基准。
原文摘要 · Abstract (English)
Generating street-view images from satellite imagery is a challenging task, particularly in maintaining accurate pose alignment and incorporating diverse environmental conditions. While diffusion models have shown promise in generative tasks, their ability to maintain strict pose alignment throughout the diffusion process is limited. In this paper, we propose a novel Iterative Homography Adjustment (IHA) scheme applied during the denoising process, which effectively addresses pose misalignment and ensures spatial consistency in the generated street-view images. Additionally, currently, available datasets for satellite-to-street-view generation are limited in their diversity of illumination and weather conditions, thereby restricting the generalizability of the generated outputs. To mitigate this, we introduce a text-guided illumination and weather-controlled sampling strategy that enables fine-grained control over the environmental factors. Extensive quantitative and qualitative evaluations demonstrate that our approach significantly improves pose accuracy and enhances the diversity and realism of generated street-view images, setting a new benchmark for satellite-to-street-view generation tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。