arXiv:2506.17759cs.CV2025-06被引 8

轻量级模型提升高光谱图像分类,少标签下表现更优

LoLA-SpecViT: Local Attention SwiGLU Vision Transformer with LoRA for Hyperspectral Imaging

  • 用局部注意力+3D卷积提取光谱空间特征,降低计算开销
  • 引入低秩适配(LoRA)仅需原参数8%即可微调,少样本效果好
  • 适合农业、遥感等真实场景,代码开源可复现

高光谱图像分类因光谱数据维度高、波段间冗余大、标注样本有限而具挑战性。现有基于Transformer的模型虽能建模全局光谱-空间依赖,但在少标签条件下仍存在扩展性与适应性不足的问题。本文提出轻量级光谱视觉变压器LoLA-SpecViT,通过3D卷积光谱前端结合局部窗口自注意力机制,增强光谱特征提取与空间一致性,同时降低计算复杂度。为提升适应性,在注意力与投影层集成低秩适配(LoRA),实现微调时参数量减少超80%。设计新型周期性学习率调度器,动态调节LoRA适应强度,提升收敛性与泛化能力。在WHU-Hi LongKou、WHU-Hi HongHu和Salinas三个基准数据集上实验表明,该模型持续优于现有最优方法,最高准确率达99.91%,参数量显著减少,且在低标签场景下更具鲁棒性。所提框架为农业、环境监测与遥感分析等实际应用提供可扩展、通用的解决方案。代码已开源。

原文摘要 · Abstract (English)

Hyperspectral image classification remains a challenging task due to the high dimensionality of spectral data, significant inter-band redundancy, and the limited availability of annotated samples. While recent transformer-based models have improved the global modeling of spectral-spatial dependencies, their scalability and adaptability under label-scarce conditions remain limited. In this work, we propose \textbf{LoLA-SpecViT}(Low-rank adaptation Local Attention Spectral Vision Transformer), a lightweight spectral vision transformer that addresses these limitations through a parameter-efficient architecture tailored to the unique characteristics of hyperspectral imagery. Our model combines a 3D convolutional spectral front-end with local window-based self-attention, enhancing both spectral feature extraction and spatial consistency while reducing computational complexity. To further improve adaptability, we integrate low-rank adaptation (LoRA) into attention and projection layers, enabling fine-tuning with over 80\% fewer trainable parameters. A novel cyclical learning rate scheduler modulates LoRA adaptation strength during training, improving convergence and generalisation. Extensive experiments on three benchmark datasets WHU-Hi LongKou, WHU-Hi HongHu, and Salinas demonstrate that LoLA-SpecViT consistently outperforms state-of-the-art baselines, achieving up to 99.91\% accuracy with substantially fewer parameters and enhanced robustness under low-label regimes. The proposed framework provides a scalable and generalizable solution for real-world HSI applications in agriculture, environmental monitoring, and remote sensing analytics. Our code is available in the following \href{https://github.com/FadiZidiDz/LoLA-SpecViT}{GitHub Repository}.

高光谱视觉transformerLoRA少样本

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。