用几何引导提升效率,实现轻量级语义分割
Breaking the Resource Wall: Geometry-Guided Sequence Modeling for Efficient Semantic Segmentation

- 设计线性复杂度的G-Mamba模块替代传统上下文建模
- 在28k次迭代下达80.8% mIoU,Cityscapes上82.3%
- 适合资源受限场景,8GB显存可稳定运行
近年来高性能语义分割依赖于大型骨干网络和高计算预算,带来显著计算开销并限制了在资源受限硬件上的应用。本文提出DGM-Net(方向几何Mamba网络),通过结构设计而非增加模型容量提升建模能力。引入方向几何Mamba(G-Mamba)——一种线性复杂度O(N)的算子,替代ASPP、PPM等传统上下文建模模块。为增强状态空间模型(SSM)中的结构感知,设计了DGM-Module,提取向心流场与拓扑骨架以指导扫描过程,提升边界保持能力。无需大规模预训练或重型骨干扩展,DGM-Net在28k次迭代内达到80.8% mIoU,Cityscapes测试集上达82.3% mIoU,ADE20K上达45.24% mIoU。模型在受限硬件下表现稳定(如8GB VRAM下批量大小为2),证明将几何引导融入SSM架构是高效语义分割的有效路径。
原文摘要 · Abstract (English)
High-performance semantic segmentation has achieved significant progress in recent years, often driven by increasingly large backbones and higher computational budgets. While effective, such approaches introduce substantial computational overhead and limit accessibility under constrained hardware settings. In this paper, we propose DGM-Net (Directional Geometric Mamba Network), an efficient architecture that improves modeling capability through structural design rather than increasing model capacity. We introduce Directional Geometric Mamba (G-Mamba), a linear-complexity O(N) operator as an alternative to conventional context modeling modules such as ASPP and PPM. To further enhance structural awareness in state space model (SSM)-based modeling, we design the DGM-Module, which extracts centripetal flow fields and topological skeletons to guide the scanning process and improve boundary preservation. Without relying on large-scale pretraining or heavy backbone scaling, DGM-Net achieves 80.8% mIoU within 28k iterations, 82.3% mIoU on Cityscapes test set, and 45.24% mIoU on ADE20K. In addition, the model maintains stable performance under constrained hardware settings (e.g., batch size of 2 on 8GB VRAM), highlighting its efficiency and practicality. These results demonstrate that incorporating geometric guidance into SSM-based architectures provides an effective and resource-efficient direction for semantic segmentation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。