TCSAFormer通过压缩注意力与双分支网络,提升医学图像分割效率与精度。
TCSAFormer: Efficient Vision Transformer with Token Compression and Sparse Attention for Medical Image Segmentation
- 用动态压缩注意力减少无关特征,降低计算复杂度。
- 在三个数据集上优于现有最优方法,参数量更少。
- 适合需要高效高精度分割的医疗影像场景。
近年来,基于Transformer的方法在医学图像分割中取得显著进展,因其能有效捕捉长距离依赖关系。然而,这类方法通常存在两大局限:一是计算复杂度随输入序列长度呈平方级增长;二是原始Transformer中的前馈网络(FFN)依赖全连接层,难以捕捉局部上下文信息和多尺度特征,影响精确语义分割。为此,本文提出一种高效的医学图像分割网络TCSAFormer。该模型采用两项关键技术:首先引入压缩注意力(CA)模块,结合令牌压缩与像素级稀疏注意力,动态聚焦每个查询最相关的键值对,通过全局剔除无关令牌并合并冗余项,显著降低计算复杂度,同时增强令牌间关系建模能力;其次,以双分支前馈网络(DBFFN)替代标准FFN,强化对局部上下文与多尺度信息的捕获,提升特征表达能力。我们在ISIC-2018、CVC-ClinicDB和Synapse三个公开数据集上进行了广泛实验,结果表明,TCSAFormer在性能上超越现有最先进方法,且计算开销更低,实现了效率与准确性的最佳平衡。
原文摘要 · Abstract (English)
In recent years, transformer-based methods have achieved remarkable progress in medical image segmentation due to their superior ability to capture long-range dependencies. However, these methods typically suffer from two major limitations. First, their computational complexity scales quadratically with the input sequences. Second, the feed-forward network (FFN) modules in vanilla Transformers typically rely on fully connected layers, which limits models' ability to capture local contextual information and multiscale features critical for precise semantic segmentation. To address these issues, we propose an efficient medical image segmentation network, named TCSAFormer. The proposed TCSAFormer adopts two key ideas. First, it incorporates a Compressed Attention (CA) module, which combines token compression and pixel-level sparse attention to dynamically focus on the most relevant key-value pairs for each query. This is achieved by pruning globally irrelevant tokens and merging redundant ones, significantly reducing computational complexity while enhancing the model's ability to capture relationships between tokens. Second, it introduces a Dual-Branch Feed-Forward Network (DBFFN) module as a replacement for the standard FFN to capture local contextual features and multiscale information, thereby strengthening the model's feature representation capability. We conduct extensive experiments on three publicly available medical image segmentation datasets: ISIC-2018, CVC-ClinicDB, and Synapse, to evaluate the segmentation performance of TCSAFormer. Experimental results demonstrate that TCSAFormer achieves superior performance compared to existing state-of-the-art (SOTA) methods, while maintaining lower computational overhead, thus achieving an optimal trade-off between efficiency and accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。