提出一种新型旋转等变卷积，提升图像分割对任意角度的鲁棒性。
PreCM: The Padding-based Rotation Equivariant Convolution Mode for Semantic Segmentation
- 通过填充设计实现旋转等变卷积，兼容多尺度与多种卷积类型。
- 在三个数据集上，平均交并比提升4.5%至10.6%，旋转差异降低3.5%以上。
- 适用于遥感、医学影像等无先验方向信息的场景，适合图像分割研究者。
语义分割是图像处理与计算机视觉的重要分支。随着深度学习的发展，各类卷积神经网络被用于像素级分类与分割任务。然而在实际应用中，成像角度往往任意，如遥感中的水体图像、医疗领域的毛细血管和息肉图像，通常缺乏先验方向信息，难以引导网络提取有效特征。此时，从多方向目标中学习特征面临挑战，因多数基于CNN的分割网络缺乏旋转等变性，易受方向干扰。为此，本文首先构建通用卷积群框架，充分挖掘方向信息并赋予网络旋转等变能力。随后，数学设计了一种基于填充的旋转等变卷积模式（PreCM），不仅适用于多尺度图像与卷积核，还可作为空洞卷积、转置卷积、非对称卷积等多种卷积的替代组件。为定量评估旋转对分割的影响，还提出了新评价指标旋转差异（RD）。在三个数据集上对六种现有分割网络进行替换实验，结果显示，其基于PreCM版本的平均交并比（IOU）分别提升6.91%、10.63%、4.53%、5.93%、7.48%、8.33%（随机角度旋转下）；平均RD值分别下降3.58%、4.56%、3.47%、3.66%、3.47%、3.43%。
原文摘要 · Abstract (English)
Semantic segmentation is an important branch of image processing and computer vision. With the popularity of deep learning, various convolutional neural networks have been proposed for pixel-level classification and segmentation tasks. In practical scenarios, however, imaging angles are often arbitrary, encompassing instances such as water body images from remote sensing and capillary and polyp images in the medical domain, where prior orientation information is typically unavailable to guide these networks to extract more effective features. In this case, learning features from objects with diverse orientation information poses a significant challenge, as the majority of CNN-based semantic segmentation networks lack rotation equivariance to resist the disturbance from orientation information. To address this challenge, this paper first constructs a universal convolution-group framework aimed at more fully utilizing orientation information and equipping the network with rotation equivariance. Subsequently, we mathematically design a padding-based rotation equivariant convolution mode (PreCM), which is not only applicable to multi-scale images and convolutional kernels but can also serve as a replacement component for various types of convolutions, such as dilated convolutions, transposed convolutions, and asymmetric convolution. To quantitatively assess the impact of image rotation in semantic segmentation tasks, we also propose a new evaluation metric, Rotation Difference (RD). The replacement experiments related to six existing semantic segmentation networks on three datasets show that, the average Intersection Over Union (IOU) of their PreCM-based versions respectively improve 6.91%, 10.63%, 4.53%, 5.93%, 7.48%, 8.33% compared to their original versions in terms of random angle rotation. And the average RD values are decreased by 3.58%, 4.56%, 3.47%, 3.66%, 3.47%, 3.43% respectively.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。