解决多数据集分割中的语义不一致问题，提升模型泛化能力。
Resolving Inconsistent Semantics in Multi-Dataset Image Segmentation
- 用语言嵌入和标签空间查询嵌入融合多数据集信息。
- 在四个基准数据集上显著优于已有方法，指标提升1.6%~12.1%。
- 适合处理跨数据集语义冲突的分割任务，尤其适用于复杂场景。
利用多个训练数据集扩展图像分割模型，有助于提升鲁棒性和语义理解能力。各数据集具有明确标注、互不重叠的掩码布局和互斥的语义定义。然而，合并训练会破坏这种一致性，导致语义冲突：例如，一个数据集中“person”与另一个数据集中“face”可能需对某些像素同时标记。现有方法在此设置下表现不佳，尤其在混合标签空间的评估中。为此，我们提出一种简单有效的多数据集训练方法，通过整合类别名称的语言嵌入与标签空间特异性查询嵌入，有效缓解语义不一致问题。该方法在存在标签空间不一致的四组基准测试中表现优异：语义分割提升1.6% mIoU，全景分割提升9.1% PQ，实例分割提升12.1% AP，新提出的PIQ指标提升3.0%。
原文摘要 · Abstract (English)
Leveraging multiple training datasets to scale up image segmentation models is beneficial for increasing robustness and semantic understanding. Individual datasets have well-defined ground truth with non-overlapping mask layouts and mutually exclusive semantics. However, merging them for multi-dataset training disrupts this harmony and leads to semantic inconsistencies; for example, the class "person" in one dataset and class "face" in another will require multilabel handling for certain pixels. Existing methods struggle with this setting, particularly when evaluated on label spaces mixed from the individual training sets. To overcome these issues, we introduce a simple yet effective multi-dataset training approach by integrating language-based embeddings of class names and label space-specific query embeddings. Our method maintains high performance regardless of the underlying inconsistencies between training datasets. Notably, on four benchmark datasets with label space inconsistencies during inference, we outperform previous methods by 1.6% mIoU for semantic segmentation, 9.1% PQ for panoptic segmentation, 12.1% AP for instance segmentation, and 3.0% in the newly proposed PIQ metric.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。