arXiv:2502.19540cs.CV2025-02被引 1

用词典+对比学习实现可解释且一致的物体解析,性能领先。

Dictionary-based Framework for Interpretable and Consistent Object Parsing

  • 基于词典的掩码变压器,每个组件对应特定语义类。
  • 在PartImageNet和Pascal-Part-108上部分mIoU分别提升2.08%和0.70%。
  • 适合关注可解释性与细粒度分割的视觉研究者。

本文提出CoCal,一种基于词典的可解释且一致的物体解析框架。该框架围绕对比组件与逻辑约束设计,重新思考现有基于聚类的掩码变压器架构。CoCal使用一组词典组件,每个组件显式关联特定语义类别,并引入分层词典结构以契合语义层次。通过同级对比组件与跨级逻辑约束的结合,实现层级内组件间的类内/类间对比,以及部件组件与对应对象组件之间的距离优化。此外,通过后处理函数确保像素分配符合“部件属于其所属对象”的逻辑。在PartImageNet和Pascal-Part-108上,CoCal分别取得2.08%和0.70%的部分mIoU提升,显著优于先前方法。同时,在物体级指标上也有明显改进,表明其不仅能提升细粒度解析,还能增强整体分割质量。

原文摘要 · Abstract (English)

In this work, we present CoCal, an interpretable and consistent object parsing framework based on dictionary-based mask transformer. Designed around Contrastive Components and Logical Constraints, CoCal rethinks existing cluster-based mask transformer architectures used in segmentation; Specifically, CoCal utilizes a set of dictionary components, with each component being explicitly linked to a specific semantic class. To advance this concept, CoCal introduces a hierarchical formulation of dictionary components that aligns with the semantic hierarchy. This is achieved through the integration of both within-level contrastive components and cross-level logical constraints. Concretely, CoCal employs a component-wise contrastive algorithm at each semantic level, enabling the contrasting of dictionary components within the same class against those from different classes. Furthermore, CoCal addresses logical concerns by ensuring that the dictionary component representing a particular part is closer to its corresponding object component than to those of other objects through a cross-level contrastive learning objective. To further enhance our logical relation modeling, we implement a post-processing function inspired by the principle that a pixel assigned to a part should also be assigned to its corresponding object. With these innovations, CoCal establishes a new state-of-the-art performance on both PartImageNet and Pascal-Part-108, outperforming previous methods by a significant margin of 2.08% and 0.70% in part mIoU, respectively. Moreover, CoCal exhibits notable enhancements in object-level metrics across these benchmarks, highlighting its capacity to not only refine parsing at a finer level but also elevate the overall quality of object segmentation.

物体解析可解释性词典模型细粒度分割

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。