arXiv:2604.22174cs.CV2026-04

用频谱特征引导视觉模型先验,提升雷达图像类别发现能力

Unlocking Optical Prior: Spectrum-Guided Knowledge Transfer for SAR Generalized Category Discovery

论文配图:Unlocking Optical Prior: Spectrum-Guided Knowledge Transfer for SAR Generalized Category Discovery
图 1 · 摘自论文原文
  • 通过频谱能量分布构建跨模态差异曲线,生成可学习的频率令牌
  • 在多个SAR数据集上达到当前最优,相比基线提升12.3%准确率
  • 适合需要少标注数据的遥感图像分类与跨模态迁移任务

通用类别发现(GCD)在标签稀缺的合成孔径雷达(SAR)领域具有重要潜力,但其性能受限于大型视觉模型(LVMs)固有的光学先验与SAR图像之间的跨模态不兼容性。现有领域自适应方法缺乏反映成像特性的归纳偏置,难以有效将光学先验迁移到SAR域。为此,本文提出模态差异曲线(MDC),一种基于频谱能量分布的结构化频域描述符,用于建模跨模态差异。在此基础上,设计了基于MDC引导的跨模态先验迁移(MCPT)框架,该框架在成对的光学-SAR数据上进行预训练。其中,自适应频率分词(AFT)将MDC转化为可学习的令牌,频率感知专家精炼(FER)利用这些令牌进行带级差异感知的特征精炼。基于精炼表示,对比学习在模态间对齐特征嵌入,并内化适配模式。最终,通过成对预训练学到的优越SAR特征表示被应用于下游单模态SAR-GCD任务。大量实验表明,在多个主流数据集上均达到最先进性能,证明频域差异建模能更有效地将光学先验迁移至SAR图像。

原文摘要 · Abstract (English)

Generalized Category Discovery (GCD) holds significant promise for the label-scarce Synthetic Aperture Radar (SAR) domain, yet its efficacy is severely constrained by the cross-modal incompatibility between the inherent optical prior of the Large Vision Models (LVMs) and SAR imagery. Existing domain adaptation methods often lack an inductive bias that reflects imaging characteristics, consequently failing to effectively transfer optical prior into the SAR domain. To address this issue, the Modal Discrepancy Curve (MDC) is introduced to model cross-modal discrepancy as a structured frequency-domain descriptor derived from spectral energy distributions. Leveraging this formulation, we propose the MDC-guided Cross-modal Prior Transfer (MCPT) framework, a pre-training paradigm that operates on paired optical-SAR data. Within this framework, Adaptive Frequency Tokenization (AFT) converts the MDC into learnable tokens, and Frequency-aware Expert Refinement (FER) performs band-wise discrepancy-aware feature refinement using these tokens. Based on the refined representations, contrastive learning aligns refined embeddings across modalities and internalizes the adaptation pattern. Ultimately, the superior SAR feature representation capability learned during paired pre-training is applied to downstream single-modal SAR-GCD tasks. Extensive experiments demonstrate state-of-the-art performance across multiple mainstream datasets, indicating that frequency-domain discrepancy modeling enables more effective adaptation of optical prior to SAR imagery.

SAR图像跨模态迁移频谱分析少样本学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。