用图神经网络提升图像分割精度,尤其擅长处理畸变和复杂边界。
Image Segmentation: Inducing graph-based learning
- 构建图像特征图的图结构,用GNN建模区域间关系
- 在三个数据集上优于U-Net、U-Net++和SwinUNet
- 适合需要长程依赖建模的场景,如自动驾驶与医学影像
本研究探索图神经网络(GNN)在多种图像模态下增强语义分割的潜力。我们在三个不同数据集上评估了一种新型基于GNN的U-Net架构:PascalVOC(自然图像分割标准基准)、WoodScape(自动驾驶常用鱼眼图像,含显著几何畸变)以及ISIC2016(皮肤病变分割的皮肤镜图像)。将提出的UNet-GNN模型与基于卷积神经网络(CNN)的U-Net、U-Net++及基于Transformer的SwinUNet进行对比。与依赖局部卷积或全局自注意力的方法不同,GNN通过图像特征的图表示显式建模区域间关系,有助于捕捉长程依赖与复杂空间结构。实验表明,该方法在多种分割挑战中表现出良好泛化能力,尤其在处理鱼眼畸变和医疗图像细微边界方面表现优异,验证了GNN在自动驾驶与医学图像分析中的应用潜力。
原文摘要 · Abstract (English)
This study explores the potential of graph neural networks (GNNs) to enhance semantic segmentation across diverse image modalities. We evaluate the effectiveness of a novel GNN-based U-Net architecture on three distinct datasets: PascalVOC, a standard benchmark for natural image segmentation, WoodScape, a challenging dataset of fisheye images commonly used in autonomous driving, introducing significant geometric distortions; and ISIC2016, a dataset of dermoscopic images for skin lesion segmentation. We compare our proposed UNet-GNN model against established convolutional neural networks (CNNs) based segmentation models, including U-Net and U-Net++, as well as the transformer-based SwinUNet. Unlike these methods, which primarily rely on local convolutional operations or global self-attention, GNNs explicitly model relationships between image regions by constructing and operating on a graph representation of the image features. This approach allows the model to capture long-range dependencies and complex spatial relationships, which we hypothesize will be particularly beneficial for handling geometric distortions present in fisheye imagery and capturing intricate boundaries in medical images. Our analysis demonstrates the versatility of GNNs in addressing diverse segmentation challenges and highlights their potential to improve segmentation accuracy in various applications, including autonomous driving and medical image analysis.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。