用点位控制生成卫星图，省去繁琐标注。
TerraDiT: Point-Conditioned Diffusion Transformer for Satellite Image Synthesis
- 以点位坐标+描述文本作为控制信号，替代复杂地图标注。
- 在多个遥感数据集上超越现有生成模型，保持高细节与地理一致性。
- 适合需要快速生成真实卫星图像的科研与城市规划人员。
我们提出TerraDiT，一种基于点条件的扩散变换器，用于文本到卫星图像的生成。现有可控卫星图像生成模型通常依赖耗时且语义受限的像素级地图标注。为解决此问题，我们引入一种新颖的点基条件框架，通过点的空间位置及其关联的文本描述来控制生成过程，提供语义丰富的控制信号。该方法实现灵活、易标注、计算简单的卫星图像生成推理。为此，我们设计了自适应局部注意力机制，根据输入点查询有效正则化注意力分数。我们系统评估了训练TerraDiT的多种领域特定设计选择，包括用于对齐的卫星图像表示以及用于条件生成的地理定位表示。实验表明，TerraDiT在生成性能上显著超越当前最先进的遥感生成模型。相关模型、数据集与代码已开源：https://github.com/mvrl/TerraDiT。
原文摘要 · Abstract (English)
We introduce TerraDiT, a diffusion transformer designed for text-to-satellite image generation with point-based control. Existing controlled satellite image generative models often require pixel-level maps that are time-consuming to acquire, yet semantically limited. To address this limitation, we introduce a novel point-based conditioning framework that controls the generation process through the spatial location of the points and the textual description associated with each point, providing semantically rich control signals. This approach enables flexible, annotation-friendly, and computationally simple inference for satellite image generation. To this end, we introduce an adaptive local attention mechanism that effectively regularizes the attention scores based on the input point queries. We systematically evaluate various domain-specific design choices for training TerraDiT, including the selection of satellite image representation for alignment and geolocation representation for conditioning. Our experiments demonstrate that TerraDiT achieves impressive generation performance, surpassing the state-of-the-art remote sensing generative models. Our models, dataset, and code are available at https://github.com/mvrl/TerraDiT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。