arXiv:2504.06205eess.IVcs.CV2025-04被引 2

HER-Seg高效分割高分辨率医学图像,显著降低计算与内存开销。

HER-Seg: Holistically Efficient Segmentation for High-Resolution Medical Images

  • 采用线性复杂度的双门控注意力机制,高效建模长距离依赖。
  • 仅需0.59GB显存训练,1024×1024图像推理仅9.39G FLOPs。
  • 适用于临床级高分辨率影像分割,适合部署在资源受限场景。

高分辨率分割对精准疾病诊断至关重要,可提取细微形态学特征。现有分层编码器-解码器框架虽适应性强,但在处理大尺寸图像时计算与内存开销巨大,限制了其在基础模型构建及临床应用中的推广。为此,我们提出一种全链路高效的高分辨率医学图像分割框架HER-Seg。首先设计计算高效的图像编码器(CE-Encoder),通过双门控线性注意力(DLA)机制实现级联标记过滤,以线性复杂度建模长程依赖并保留关键信息。随后引入内存高效的掩码解码器(ME-Decoder),摒弃层级结构,利用跨尺度分割解码提升效率。大量实验表明,HER-Seg在2D、3D及视频医学图像分割任务中均超越现有方法。尤其在1024×1024图像上,训练仅需0.59GB GPU显存,推理仅9.39G FLOPs,展现出卓越的计算与内存效率。代码已开源:https://github.com/xq141839/HER-Seg。

原文摘要 · Abstract (English)

High-resolution segmentation is critical for precise disease diagnosis by extracting fine-grained morphological details. Existing hierarchical encoder-decoder frameworks have demonstrated remarkable adaptability across diverse medical segmentation tasks. While beneficial, they usually require the huge computation and memory cost when handling large-size segmentation, which limits their applications in foundation model building and real-world clinical scenarios. To address this limitation, we propose a holistically efficient framework for high-resolution medical image segmentation, called HER-Seg. Specifically, we first devise a computation-efficient image encoder (CE-Encoder) to model long-range dependencies with linear complexity while maintaining sufficient representations. In particular, we introduce the dual-gated linear attention (DLA) mechanism to perform cascaded token filtering, selectively retaining important tokens while ignoring irrelevant ones to enhance attention computation efficiency. Then, we introduce a memory-efficient mask decoder (ME-Decoder) to eliminate the demand for the hierarchical structure by leveraging cross-scale segmentation decoding. Extensive experiments reveal that HER-Seg outperforms state-of-the-arts in high-resolution medical 2D, 3D and video segmentation tasks. In particular, our HER-Seg requires only 0.59GB training GPU memory and 9.39G inference FLOPs per 1024$\times$1024 image, demonstrating superior memory and computation efficiency. The code is available at https://github.com/xq141839/HER-Seg.

医学图像分割高效模型注意力机制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。