让雷达、多光谱图像和文本在统一空间里互查,提升灾情监测效率。
CLOSP: A Unified Semantic Space for SAR, MSI, and Text in Remote Sensing
- 用文本作桥梁,把不同传感器图像对齐到同一语义空间。
- 检索性能比现有模型提升54%,在64.7万张图上验证有效。
- 加入地理坐标后可精准查找特定地点的灾害与罕见地貌。
从海量卫星数据中快速检索相关影像对灾害响应和长期气候监测至关重要。然而,现有文本到图像检索系统大多仅限于可见光数据,无法利用合成孔径雷达(SAR)的全天候结构敏感性或多光谱图像的光谱特征。为此,我们构建了CrisisLandMark,一个包含超过647,000张哨兵-1 SAR与哨兵-2多光谱图像的大型语料库,其文本标注涵盖土地覆盖、土地利用及危机事件,来自权威系统(CORINE和Dynamic World)与特定危机来源。我们提出CLOSP(对比语言光学-雷达预训练)框架,通过文本将未配对的光学与SAR图像映射至统一嵌入空间。实验表明,CLOSP在nDGC@1000指标上相较现有模型提升54%。此外,统一训练策略通过间接交互,将光学域丰富的语义知识迁移至难以解释的SAR图像。进一步地,引入地理坐标的GeoCLOSP在通用语义任务与特定地点灾情事件、稀有地理特征检索间实现权衡:前者表现优异,后者成为专业专家。本工作凸显融合多源传感器数据与地理上下文对释放遥感档案潜力的关键作用。
原文摘要 · Abstract (English)
Retrieving relevant imagery from vast satellite archives is crucial for applications like disaster response and long-term climate monitoring. However, most text-to-image retrieval systems are limited to RGB data, failing to exploit the unique physical information captured by other sensors, such as the all-weather structural sensitivity of Synthetic Aperture Radar (SAR) or the spectral signatures in optical multispectral data. To bridge this gap, we introduce CrisisLandMark, a new large-scale corpus of over 647,000 Sentinel-1 SAR and Sentinel-2 multispectral images paired with structured textual annotations for land cover, land use, and crisis events harmonized from authoritative land cover systems (CORINE and Dynamic World) and crisis-specific sources. We then present CLOSP (Contrastive Language Optical SAR Pretraining), a novel framework that uses text as a bridge to align unpaired optical and SAR images into a unified embedding space. Our experiments show that CLOSP achieves a new state-of-the-art, improving retrieval nDGC@1000 by 54% over existing models. Additionally, we find that the unified training strategy overcomes the inherent difficulty of interpreting SAR imagery by transferring rich semantic knowledge from the optical domain with indirect interaction. Furthermore, GeoCLOSP, which integrates geographic coordinates into our framework, creates a powerful trade-off between generality and specificity: while the CLOSP excels at general semantic tasks, the GeoCLOSP becomes a specialized expert for retrieving location-dependent crisis events and rare geographic features. This work highlights that the integration of diverse sensor data and geographic context is essential for unlocking the full potential of remote sensing archives.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。