用分阶段注意力提升遥感图像语义分割精度
Context-Aware Semantic Segmentation via Stage-Wise Attention
- 双分支结构结合高低分辨率特征，分阶段注入上下文信息
- 在FLAIR-HUB数据集上达66.37% mIoU，边界更清晰
- 适用于遥感与医学超高分辨率分割，代码已开源
语义超高分辨率（UHR）图像分割在航空测绘和环境监测中至关重要。基于Transformer的模型在此场景下面临挑战，因内存随标记数呈平方增长，限制了空间分辨率或上下文范围。我们提出CASWiT（上下文感知分阶段变压器），一种基于Swin的双分支架构，通过轻量级分阶段交叉注意力将低分辨率上下文信息注入细粒度高分辨率特征。为增强跨尺度学习，还设计了一种类似SimMIM的预训练策略，基于高分辨率图像的掩码重建。在大规模FLAIR-HUB航空数据集上的大量实验表明，该方法有效。在仅使用RGB的UHR协议下，搭配SegFormer解码器，CASWiT达到66.37% mIoU，优于强基线并提升边界质量；在URUR基准上，官方评估协议下达49.2% mIoU，且在医学超分辨率分割任务中具有良好的迁移能力。代码与预训练模型已在Hugging Face公开。
原文摘要 · Abstract (English)
Semantic ultra-high-resolution (UHR) image segmentation is essential in remote sensing applications such as aerial mapping and environmental monitoring. Transformer-based models remain challenging in this setting because memory grows quadratically with the number of tokens, limiting either spatial resolution or contextual scope. We introduce CASWiT (Context-Aware Stage-Wise Transformer), a dual-branch Swin-based architecture that injects low-resolution contextual information into fine-grained high-resolution features through lightweight stage-wise cross-attention. To strengthen cross-scale learning, we also propose a SimMIM-style pretraining strategy based on masked reconstruction of the high-resolution image. Extensive experiments on the large-scale FLAIR-HUB aerial dataset demonstrate the effectiveness of CASWiT. Under our RGB-only UHR protocol, CASWiT reaches 66.37% mIoU with a SegFormer decoder, improving over strong RGB baselines while also improving boundary quality. On the URUR benchmark, CASWiT reaches 49.2% mIoU under the official evaluation protocol, and it also transfers effectively to medical UHR segmentation benchmarks. Code and pretrained models are available at https://huggingface.co/collections/heig-vd-geo/caswit
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。