提出双模块提升图像语义分割精度,尤其改善边界区域表现。
A Deep Semantic Segmentation Network with Semantic and Contextual Refinements
- 设计语义修正模块,通过像素级偏移优化上采样特征对齐。
- 引入上下文修正模块,融合多阶段特征增强全局语义理解。
- 轻量模型在Cityscapes上达82.5% mIoU,仅137.9 GFLOPs。
语义分割是多媒体处理的基础任务,常用于图像与视频内容的分析与编辑。现有方法通过逐步降低特征图空间分辨率提取语义信息,但在恢复高分辨率时易产生特征错位问题。本文设计语义修正模块(SRM),通过高分辨率特征和邻近偏移引导,为上采样特征图中每个像素学习变换偏移,提升边界区域的语义表示。同时提出上下文修正模块(CRM),跨空间与通道维度捕捉全局上下文信息,通过融合骨干网络四阶段的语义图增强通道上下文。在Cityscapes、Bdd100K和ADE20K三个主流数据集上验证有效性,性能优于当前最优方法。进一步将模块应用于轻量级网络,在Cityscapes验证集上实现82.5% mIoU,仅需137.9 GFLOPs。
原文摘要 · Abstract (English)
Semantic segmentation is a fundamental task in multimedia processing, which can be used for analyzing, understanding, editing contents of images and videos, among others. To accelerate the analysis of multimedia data, existing segmentation researches tend to extract semantic information by progressively reducing the spatial resolutions of feature maps. However, this approach introduces a misalignment problem when restoring the resolution of high-level feature maps. In this paper, we design a Semantic Refinement Module (SRM) to address this issue within the segmentation network. Specifically, SRM is designed to learn a transformation offset for each pixel in the upsampled feature maps, guided by high-resolution feature maps and neighboring offsets. By applying these offsets to the upsampled feature maps, SRM enhances the semantic representation of the segmentation network, particularly for pixels around object boundaries. Furthermore, a Contextual Refinement Module (CRM) is presented to capture global context information across both spatial and channel dimensions. To balance dimensions between channel and space, we aggregate the semantic maps from all four stages of the backbone to enrich channel context information. The efficacy of these proposed modules is validated on three widely used datasets-Cityscapes, Bdd100K, and ADE20K-demonstrating superior performance compared to state-of-the-art methods. Additionally, this paper extends these modules to a lightweight segmentation network, achieving an mIoU of 82.5% on the Cityscapes validation set with only 137.9 GFLOPs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。