用极化散射机制增强分割模型,提升雷达图像理解能力
PolSAM: Polarimetric Scattering Mechanism Informed Segment Anything Model
- 引入可解释的微波视觉数据表示,融合极化分解与语义关联
- 在PhySAR-Seg上实现更优分割精度,存储减少37%,推理提速28%
- 适合遥感图像分割、极化SAR分析等领域的研究者使用
极化合成孔径雷达(PolSAR)数据因其丰富的复杂特性带来独特挑战。现有数据表示形式如复数数据、极化特征和幅值图像普遍存在可用性差、可解释性低和数据完整性不足的问题。多数PolSAR特征提取网络规模较小,难以有效捕捉特征。为此,我们提出极化散射机制启发的Segment Anything Model(PolSAM),一种增强型分段通用模型,融合领域特定的散射特性与新型提示生成策略。PolSAM引入微波视觉数据(MVD),一种基于极化分解与语义相关性的轻量且可解释的数据表示。提出两个核心组件:特征级融合提示(FFP),通过融合伪彩色SAR图像的视觉令牌与MVD解决冻结SAM编码器中的模态不兼容问题;语义级融合提示(SFP),利用语义信息优化稀疏与密集分割提示。在PhySAR-Seg数据集上的实验表明,PolSAM显著优于现有基于SAM及多模态融合模型,提升分割精度,减少37%数据存储,加速28%推理时间。源代码与数据集将公开于https://github.com/XAI4SAR/PolSAM。
原文摘要 · Abstract (English)
PolSAR data presents unique challenges due to its rich and complex characteristics. Existing data representations, such as complex-valued data, polarimetric features, and amplitude images, are widely used. However, these formats often face issues related to usability, interpretability, and data integrity. Most feature extraction networks for PolSAR are small, limiting their ability to capture features effectively. To address these issues, We propose the Polarimetric Scattering Mechanism-Informed SAM (PolSAM), an enhanced Segment Anything Model (SAM) that integrates domain-specific scattering characteristics and a novel prompt generation strategy. PolSAM introduces Microwave Vision Data (MVD), a lightweight and interpretable data representation derived from polarimetric decomposition and semantic correlations. We propose two key components: the Feature-Level Fusion Prompt (FFP), which fuses visual tokens from pseudo-colored SAR images and MVD to address modality incompatibility in the frozen SAM encoder, and the Semantic-Level Fusion Prompt (SFP), which refines sparse and dense segmentation prompts using semantic information. Experimental results on the PhySAR-Seg datasets demonstrate that PolSAM significantly outperforms existing SAM-based and multimodal fusion models, improving segmentation accuracy, reducing data storage, and accelerating inference time. The source code and datasets will be made publicly available at https://github.com/XAI4SAR/PolSAM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。