arXiv:2509.09429cs.CVcs.LG2025-09TPAMI被引 5

解决自监督密集表征学习中的语义分散问题,提升图像块一致性。

Semantic Concentration for Self-Supervised Dense Representations Learning

  • 通过提取块对应关系,打破严格空间对齐限制。
  • 设计抗噪排序损失,提升模型对伪标签噪声的鲁棒性。
  • 引入对象感知过滤器,增强复杂场景下的语义区分能力。

图像级自监督学习(SSL)取得显著进展,但图像块的密集表征学习仍具挑战。主流方法存在同一实例内块分布过散的问题,损害密集任务性能。本文揭示图像级SSL通过隐式语义集中避免过分散:非严格空间对齐保证实例内一致性,共享模式(同类实例在输入空间中的相似部分)保证跨图一致性。然而这些机制因空间敏感性和复杂场景数据难以用于密集SSL。为此,本文提出显式语义集中策略:首先,为打破严格空间对齐,提出蒸馏块对应关系;针对噪声和不平衡伪标签,设计抗噪排序损失,将平均精度(AP)损失扩展至连续目标,利用其决策无关与自适应聚焦特性防止学生模型被误导。其次,为从复杂场景中区分共享模式,提出对象感知过滤器,通过交叉注意力将块表示为可学习的对象原型。最后,跨多种任务的实证研究充分验证了方法有效性。代码已开源:https://github.com/KID-7391/CoTAP。

原文摘要 · Abstract (English)

Recent advances in image-level self-supervised learning (SSL) have made significant progress, yet learning dense representations for patches remains challenging. Mainstream methods encounter an over-dispersion phenomenon that patches from the same instance/category scatter, harming downstream performance on dense tasks. This work reveals that image-level SSL avoids over-dispersion by involving implicit semantic concentration. Specifically, the non-strict spatial alignment ensures intra-instance consistency, while shared patterns, i.e., similar parts of within-class instances in the input space, ensure inter-image consistency. Unfortunately, these approaches are infeasible for dense SSL due to their spatial sensitivity and complicated scene-centric data. These observations motivate us to explore explicit semantic concentration for dense SSL. First, to break the strict spatial alignment, we propose to distill the patch correspondences. Facing noisy and imbalanced pseudo labels, we propose a noise-tolerant ranking loss. The core idea is extending the Average Precision (AP) loss to continuous targets, such that its decision-agnostic and adaptive focusing properties prevent the student model from being misled. Second, to discriminate the shared patterns from complicated scenes, we propose the object-aware filter to map the output space to an object-based space. Specifically, patches are represented by learnable prototypes of objects via cross-attention. Last but not least, empirical studies across various tasks soundly support the effectiveness of our method. Code is available in https://github.com/KID-7391/CoTAP.

自监督学习密集表征语义集中

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。