arXiv:2607.12782cs.CV2026-07

保留全波段信息,高效适配遥感图像分类的多分支微调框架

MBTI: A Multi-Branch Efficient Fine-Tuning Framework for Hyperspectral Image Classification with Foundation Models

论文配图:MBTI: A Multi-Branch Efficient Fine-Tuning Framework for Hyperspectral Image Classification with Foundation Models
图 1 · 摘自论文原文
  • 多分支结构分块处理光谱数据,避免信息丢失
  • 仅2.36%参数可训练,实现高效微调
  • 适合波段差异大的遥感任务,对小样本友好

高光谱基础模型通过大规模无标签数据学习可迁移的光谱-空间表征,为少量标注样本的下游分类任务提供有效方案。然而传感器间光谱波段配置差异大,直接迁移困难。现有方法常通过压缩、选择或重整形光谱以匹配模型输入,可能丢弃有用信息并破坏局部光谱连续性。为此,我们提出MBTI:一种多分支高效微调框架,可在保持全波段信息的前提下适配高光谱图像分类任务。首先,设计光谱连续性保持的多分支预处理策略,将原始高光谱图像划分为多个连续波段子集,并引入波段复用机制,避免无效填充和冗余损失;其次,在各分支中独立插入低秩适应(LoRA)模块,使不同波段区间学习任务特定判别特征,同时冻结大部分预训练参数;最后,采用多分支通道注意力融合模块,自适应地校准并整合各分支特征。在三个公开高光谱数据集上的实验表明,MBTI性能优于或媲美主流方法。在最终秩8配置下,仅约2.33%–2.36%的参数可训练。代码将发布于https://github.com/Azhenmiddleblock/MBTI/tree/main。

原文摘要 · Abstract (English)

Hyperspectral foundation models learn transferable spectral-spatial representations from large-scale unlabeled data. They provide an effective paradigm for adapting to downstream hyperspectral image (HSI) classification tasks with limited labeled samples. However, spectral band configurations vary substantially across sensors, which makes direct model transfer difficult. Existing adaptation strategies often compress, select, or reshape the original spectra to match model-specific input requirements. These operations may discard useful spectral information and weaken local spectral continuity. To address this problem, we propose MBTI, a Multi-Branch efficient fine-tuning framework for Hyperspectral Image classification. MBTI adapts hyperspectral foundation models to downstream classification tasks while preserving full-band spectral information. First, we introduce a spectral-continuity-preserving multi-branch preprocessing strategy. The original HSI is divided into multiple continuous spectral subsets, and a band reuse mechanism is used when the remaining bands cannot form a complete branch. This avoids invalid padding and unnecessary spectral loss. Second, independent Low-Rank Adaptation (LoRA) modules are inserted into each branch. They enable different spectral intervals to learn task-specific discriminative features while keeping most pre-trained parameters frozen. Finally, a multi-branch channel attention fusion module adaptively recalibrates and integrates features from all spectral branches. Experiments on three public hyperspectral datasets show that MBTI achieves competitive and superior performance compared with representative classification methods. Under the final rank-8 configuration, only about 2.33\%--2.36\% of the parameters are trainable. The code will be available at https://github.com/Azhenmiddleblock/MBTI/tree/main.

高光谱图像多分支高效微调光谱连续性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。