解决光学与雷达图像识别差异，提升海上船舶跨模态识别准确率
MOS: Mitigating Optical-SAR Modality Gap for Cross-Modal Ship Re-Identification
- 通过特征对齐和扩散模型生成跨模态样本，缩小光学与雷达图像差异
- 在三种场景下分别提升3.0%、6.2%和16.4%的识别准确率
- 适合海洋监控、智能航运等需要跨模态识别的应用场景
光学与合成孔径雷达（SAR）图像之间的跨模态船舶重识别（ReID）在海事情报与监视中日益重要，但两者间存在显著模态差距。为此，本文提出MOS框架，通过模态一致特征学习与跨模态数据生成融合，缓解该差距。首先，模态一致表示学习（MCRL）利用去噪SAR处理和类别级模态对齐损失，对齐跨模态身份特征分布；其次，跨模态数据生成与特征融合（CDGF）采用布朗桥扩散模型生成跨模态样本，并在推理阶段融合原始特征以增强对齐性与判别力。在HOSS ReID数据集上的实验表明，MOS在所有评估协议下均显著优于现有方法，分别在ALL to ALL、Optical to SAR、SAR to Optical设置下实现R1准确率提升+3.0%、+6.2%、+16.4%。
原文摘要 · Abstract (English)
Cross-modal ship re-identification (ReID) between optical and synthetic aperture radar (SAR) imagery has recently emerged as a critical yet underexplored task in maritime intelligence and surveillance. However, the substantial modality gap between optical and SAR images poses a major challenge for robust identification. To address this issue, we propose MOS, a novel framework designed to mitigate the optical-SAR modality gap and achieve modality-consistent feature learning for optical-SAR cross-modal ship ReID. MOS consists of two core components: (1) Modality-Consistent Representation Learning (MCRL) applies denoise SAR image procession and a class-wise modality alignment loss to align intra-identity feature distributions across modalities. (2) Cross-modal Data Generation and Feature fusion (CDGF) leverages a brownian bridge diffusion model to synthesize cross-modal samples, which are subsequently fused with original features during inference to enhance alignment and discriminability. Extensive experiments on the HOSS ReID dataset demonstrate that MOS significantly surpasses state-of-the-art methods across all evaluation protocols, achieving notable improvements of +3.0%, +6.2%, and +16.4% in R1 accuracy under the ALL to ALL, Optical to SAR, and SAR to Optical settings, respectively. The code and trained models will be released upon publication.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。