arXiv:2511.22958cs.CV2025-11中稿 · able submissions

为太阳观测数据设计对比预训练模型,提升图像分析效果。

Contrastive Heliophysical Image Pretraining for Solar Dynamics Observatory Records

  • 构建多粒度对比学习框架,对齐不同仪器的时空特征。
  • 在低标注数据下仍达顶尖性能,跨模态翻译与耀斑分类均提升显著。
  • 适合太阳物理研究者快速部署,降低计算成本,提高标签效率。

深度学习已革新太阳图像分析,但多数方法从头训练特定任务编码器,或依赖忽略太阳观测特性的自然图像预训练。我们提出SolarCHIP,一套专为多仪器太阳动力学观测卫星(SDO)数据设计的对比预训练视觉主干网络。SolarCHIP解决三大挑战:AIA与HMI仪器间的多模态感知、缓慢时序演化导致的类间区分弱、以及稀疏活动信号带来的类内高变异性。预训练框架采用多粒度对比目标,联合对齐:(1) 同时间的AIA-HMI图像对中的全局类别标记以增强时序判别力;(2) 固定空间位置的局部图块标记以实现位置一致且模态不变特征;(3) 单样本内不同空间位置的图块以保留细粒度空间结构。我们训练了基于CNN和视觉变换器的自编码器,并在两个下游任务上验证其有效性:通过ControlNet实现HMI与AIA波段间的跨模态转换,以及全盘耀斑分类。实验表明,SolarCHIP在两项任务中均达到当前最优性能,尤其在标注数据稀缺的低资源场景中表现突出。消融实验确认每项对比组件在不同粒度上贡献关键判别能力。通过公开预训练权重与训练代码,我们为日地物理学界提供了一个可即插即用的特征提取器,显著降低计算开销,提升标签效率,并为多样化的太阳成像应用建立可复用基础。

原文摘要 · Abstract (English)

Deep learning has revolutionized solar image analysis, yet most approaches train task-specific encoders from scratch or rely on natural-image pretraining that ignores the unique characteristics of Solar Dynamics Observatory (SDO) data. We introduce SolarCHIP, a family of contrastively pretrained visual backbones tailored to multi-instrument SDO observations. SolarCHIP addresses three key challenges in solar imaging: multimodal sensing across AIA and HMI instruments, weak inter-class separability due to slow temporal evolution, and strong intra-class variability with sparse activity signals. Our pretraining framework employs a multi-granularity contrastive objective that jointly aligns (1) global class tokens across co-temporal AIA-HMI pairs to enhance temporal discrimination, (2) local patch tokens at fixed spatial indices to enforce position-consistent, modality-invariant features, and (3) intra-sample patches across different spatial locations to preserve fine-grained spatial structure. We train both CNN- and Vision Transformer-based autoencoders and demonstrate their effectiveness on two downstream tasks: cross-modal translation between HMI and AIA passbands via ControlNet, and full-disk flare classification. Experimental results show that SolarCHIP achieves state-of-the-art performance across both tasks, with particularly strong gains in low-resource settings where labeled data is limited. Ablation studies confirm that each contrastive component contributes essential discriminative capacity at different granularities. By publicly releasing pretrained weights and training code, we provide the heliophysics community with a practical, plug-and-play feature extractor that reduces computational requirements, improves label efficiency, and establishes a reusable foundation for diverse solar imaging applications.

太阳图像对比学习预训练多模态

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。