arXiv:2604.23622cs.CV2026-04被引 36

融合CNN与Transformer,提升高光谱图像分类精度。

A Synergistic CNN-Transformer Network with Pooling Attention Fusion for Hyperspectral Image Classification

论文配图:A Synergistic CNN-Transformer Network with Pooling Attention Fusion for Hyperspectral Image Classification
图 1 · 摘自论文原文
  • 双分支结构并行提取空间与光谱特征。
  • 跨层特征融合减少信息丢失,准确率最高达98.7%。
  • 适合遥感图像分析与环境监测领域研究者。

高光谱图像(HSI)分类任务中,每个像素被归类为特定地物或材料类别。卷积神经网络(CNN)和变压器(Transformer)被广泛用于提取局部和非局部特征。近期工作采用多尺度视觉变压器(ViT)以增强光谱特征捕捉并取得良好效果。然而,现有方法在有效联合利用空间-光谱信息以及在传播过程中保持层间信息方面仍面临挑战。为此,我们提出一种协同的CNN-Transformer网络,结合池化注意力融合机制用于高光谱图像分类,通过CNN和ViT分别处理空间与光谱特征。具体地,设计了孪生分支特征提取(TBFE)模块,采用3D和2D卷积并行提取光谱与空间特征;提出混合池化注意力(HPA)模块聚合空间注意力;采用级联变压器编码器进行全局光谱特征提取,并设计了一种简单高效的跨层特征融合(CFF)模块,以减少前层关键信息的损失。在多个代表性数据集上进行了大量实验,结果表明所提方法优于当前最先进模型。代码已公开于 https://github.com/chenpeng052/SCT-Net.git。

原文摘要 · Abstract (English)

In the hyperspectral image (HSI) classification task, each pixel is categorized into a specific land-cover category or material. Convolutional neural networks (CNNs) and transformers have been widely used to extract local and non-local features in HSI classification. Recent works have utilized a multi-scale vision transformer (ViT) to enhance spectral feature capture and yield promising results. However, most existing methods still face challenges in the effective joint use of spatial-spectral information and in preserving information across layers during the propagation process. To address these issues, we propose a synergistic CNN-Transformer network with pooling attention fusion for HSI classification, which collaboratively utilizes CNNs and ViT to process spatial and spectral features separately. Specifically, we propose a Twin-Branch Feature Extraction (TBFE) module, which employs 3D and 2D convolution in parallel to comprehensively extract spectral and spatial features from HSI. A hybrid pooling attention (HPA) module is designed to aggregate spatial attention. Moreover, a cascade transformer encoder is employed for global spectral feature extraction, and a simple yet efficient cross-layer feature fusion (CFF) module is designed to reduce the loss of crucial information in the previous network layers. Extensive experiments are conducted on several representative datasets to demonstrate the superior performance of our proposed method compared to the state-of-the-art works. Code is available at https://github.com/chenpeng052/SCT-Net.git.

高光谱CNNTransformer遥感

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。