通过添加结构线索和低层特征引导，显著提升细粒度场景分割精度
OLAF: A Plug-and-Play Framework for Enhanced Multi-object Multi-part Scene Parsing
- 输入融合物体结构线索（前景/背景、边缘），并用权重自适应让预训练模型稳定处理
- 在Pascal-Parts上实现3.3~4.0的mIoU提升，尤其对小部件分割效果显著
- 兼容多种网络架构，适合需要高精度细粒度分割的研究与应用
多对象多部件场景分割任务复杂度随部件粒度和物体数量呈指数增长。为应对该挑战，我们提出一种即插即用框架OLAF。首先，在输入（RGB）基础上增加包含物体结构线索的通道（前景/背景掩码、边界边缘掩码）。我们提出一种权重自适应技术，使常规（RGB）预训练模型能稳定处理五通道输入。此外，引入称为LDF的编码器模块，提供低层密集特征引导，有效辅助小部件分割。OLAF在最先进模型基础上实现显著提升：Pascal-Parts-58提升3.3，Pascal-Parts-108提升3.5，Pascal-Parts-201提升4.0。实验表明，OLAF具备广泛适用性，可跨多种架构（CNN、U-Net、Transformer）和数据集带来性能增益。代码已开源于olafseg.github.io。
原文摘要 · Abstract (English)
Multi-object multi-part scene segmentation is a challenging task whose complexity scales exponentially with part granularity and number of scene objects. To address the task, we propose a plug-and-play approach termed OLAF. First, we augment the input (RGB) with channels containing object-based structural cues (fg/bg mask, boundary edge mask). We propose a weight adaptation technique which enables regular (RGB) pre-trained models to process the augmented (5-channel) input in a stable manner during optimization. In addition, we introduce an encoder module termed LDF to provide low-level dense feature guidance. This assists segmentation, particularly for smaller parts. OLAF enables significant mIoU gains of $\mathbf{3.3}$ (Pascal-Parts-58), $\mathbf{3.5}$ (Pascal-Parts-108) over the SOTA model. On the most challenging variant (Pascal-Parts-201), the gain is $\mathbf{4.0}$. Experimentally, we show that OLAF's broad applicability enables gains across multiple architectures (CNN, U-Net, Transformer) and datasets. The code is available at olafseg.github.io
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。