用预训练扩散模型提取遥感图像特征，少标签也能高效分类。
Label-Efficient Hyperspectral Image Classification via Spectral FiLM Modulation of Low-Level Pretrained Diffusion Features
- 从自然图像扩散模型中提取低层空间特征，适配高光谱图像结构。
- 仅用稀疏标签，在两个数据集上超越现有最优方法。
- 轻量级谱-空融合模块，适合标注稀缺的遥感任务。
高光谱成像（HSI）可实现精细地物分类，但面临空间分辨率低和标注稀疏的挑战。本文提出一种标签高效的框架，利用在自然图像上预训练的冻结扩散模型的空间特征。通过在早期去噪步骤中提取高分辨率解码器层的低层表示，这些特征能有效迁移至纹理稀疏的高光谱图像。为融合光谱与空间信息，引入轻量级FiLM融合模块，基于光谱线索自适应调制冻结的空间特征，实现稀疏监督下的鲁棒多模态学习。在两个近期高光谱数据集上的实验表明，本方法仅使用提供的稀疏训练标签，便优于现有最先进方法。消融实验进一步验证了扩散模型特征与谱感知融合的有效性。结果表明，预训练扩散模型可支持遥感及更广泛科学成像任务中的领域无关、标签高效表示学习。
原文摘要 · Abstract (English)
Hyperspectral imaging (HSI) enables detailed land cover classification, yet low spatial resolution and sparse annotations pose significant challenges. We present a label-efficient framework that leverages spatial features from a frozen diffusion model pretrained on natural images. Our approach extracts low-level representations from high-resolution decoder layers at early denoising timesteps, which transfer effectively to the low-texture structure of HSI. To integrate spectral and spatial information, we introduce a lightweight FiLM-based fusion module that adaptively modulates frozen spatial features using spectral cues, enabling robust multimodal learning under sparse supervision. Experiments on two recent hyperspectral datasets demonstrate that our method outperforms state-of-the-art approaches using only the provided sparse training labels. Ablation studies further highlight the benefits of diffusion-derived features and spectral-aware fusion. Overall, our results indicate that pretrained diffusion models can support domain-agnostic, label-efficient representation learning for remote sensing and broader scientific imaging tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。