通过跨阶段传播注意力，大幅降低语义分割计算量
Cross-Stage Attention Propagation for Efficient Semantic Segmentation
- 在深层特征上计算注意力，并传递到浅层，避免重复计算
- 在ADE20K上达42.9% mIoU，仅需5.5 GFLOPs
- 适合资源受限场景下的高效语义分割应用
近期轻量化语义分割方法通过结合紧凑主干网络与高效解码头取得了显著进展。然而，大多数多尺度解码器在每个特征尺度上独立计算注意力，导致大量冗余，因为不同尺度间的注意力分布具有强相关性。本文提出跨阶段注意力传播（CSAP）解码框架：仅在最深层特征尺度上计算注意力，并将生成的注意力图传播至浅层阶段，完全跳过这些阶段的查询-键值计算。该设计在保持多尺度上下文推理能力的同时，显著降低解码器计算成本。CSAP-Tiny 在 ADE20K 上实现 42.9% mIoU，仅需 5.5 GFLOPs；在 Cityscapes 上达 80.5%，耗时 21.5 GFLOPs；在 COCO-Stuff 164K 上达 40.9%，仅需 5.5 GFLOPs。其性能超越 SegNeXt-Tiny +1.8% mIoU，同时减少 16.8% 浮点运算量。
原文摘要 · Abstract (English)
Recent lightweight semantic segmentation methods have made significant progress by combining compact backbones with efficient decoder heads. However, most multi-scale decoders compute attention independently at each feature scale, introducing substantial redundancy since the resulting attention distributions across scales are strongly correlated. We propose Cross-Stage Attention Propagation (CSAP), a decoder framework that computes attention at the deepest feature scale and propagates the resulting attention maps to shallower stages, bypassing query-key computation at those stages entirely. This design preserves multi-scale contextual reasoning while substantially reducing the decoder's computational cost. CSAP-Tiny achieves 42.9% mIoU on ADE20K with only 5.5 GFLOPs, 80.5% on Cityscapes with 21.5 GFLOPs, and 40.9% on COCO-Stuff 164K with 5.5 GFLOPs, surpassing SegNeXt-Tiny by +1.8% on ADE20K while requiring 16.8% fewer floating-point operations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。