轻量级网络ECMNet融合CNN与Mamba，提升语义分割精度与效率。
ECMNet:Lightweight Semantic Segmentation with Efficient CNN-Mamba Network
- 基于胶囊框架融合CNN与Mamba，互补解决局部与全局建模短板。
- 在Cityscapes和CamVid上分别达到70.6%和73.6% mIoU，参数仅0.87M。
- 适合边缘设备部署，兼顾高精度与低计算开销，适用于实时分割场景。
过去十年，卷积神经网络（CNN）与Transformer在语义分割任务中广泛应用。尽管结合了两者性能显著提升，但全局上下文建模仍不足。近期，Mamba在视觉任务中展现出建模长距离依赖的潜力。本文提出一种轻量级高效CNN-Mamba网络ECMNet，用于语义分割。该模型通过基于胶囊的框架巧妙融合CNN与Mamba，弥补二者短板。设计增强双注意力模块（EDAB）实现轻量瓶颈；提出多尺度注意力单元（MSAU），集成多尺度、空间与通道特征聚合；构建Mamba增强特征融合模块（FFM），有效融合多层级特征，显著提升分割精度。在两个代表性数据集上的大量实验表明，所提模型在精度与效率间取得优异平衡：在Cityscapes测试集上达70.6% mIoU，CamVid上达73.6% mIoU，仅需0.87M参数与8.27G FLOPs（单块RTX 3090 GPU平台）。
原文摘要 · Abstract (English)
In the past decade, Convolutional Neural Networks (CNNs) and Transformers have achieved wide applicaiton in semantic segmentation tasks. Although CNNs with Transformer models greatly improve performance, the global context modeling remains inadequate. Recently, Mamba achieved great potential in vision tasks, showing its advantages in modeling long-range dependency. In this paper, we propose a lightweight Efficient CNN-Mamba Network for semantic segmentation, dubbed as ECMNet. ECMNet combines CNN with Mamba skillfully in a capsule-based framework to address their complementary weaknesses. Specifically, We design a Enhanced Dual-Attention Block (EDAB) for lightweight bottleneck. In order to improve the representations ability of feature, We devise a Multi-Scale Attention Unit (MSAU) to integrate multi-scale feature aggregation, spatial aggregation and channel aggregation. Moreover, a Mamba enhanced Feature Fusion Module (FFM) merges diverse level feature, significantly enhancing segmented accuracy. Extensive experiments on two representative datasets demonstrate that the proposed model excels in accuracy and efficiency balance, achieving 70.6% mIoU on Cityscapes and 73.6% mIoU on CamVid test datasets, with 0.87M parameters and 8.27G FLOPs on a single RTX 3090 GPU platform.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。