无需训练即可提升物体中心掩码质量，解决碎片化与边界泄漏问题。
SSR: Similarity-Shift Refinement for Training-Free Object-Centric Masks
- 基于自注意力值空间相似性变化，构建稀疏亲和图进行掩码优化。
- 在24个组合中平均提升8.5个百分点的调整兰德指数。
- 适用于多种场景，尤其适合不想重训模型的研究者使用。
物体中心模型常产生碎片化掩码、边界渗漏和区域错误合并。本文提出无需训练的后处理方法SSR，利用冻结的自监督视觉变换器，通过测量自注意力值聚合前后像素块间相似性的变化，保留正向增强的关系，构建稀疏亲和图。该图在单步内传播初始软槽分配，无需重新训练或修改模型。在自然图像、合成视频和真实视频基准上，SSR在所有24个模型-数据集组合中均提升了全像素调整兰德指数，平均提升8.5个百分点。消融实验表明，值空间相似性变化优于查询和键空间变体及静态变压器亲和度。然而，在纹理密集场景中可能导致视觉相似区域过度合并。总体而言，SSR为无需训练的物体中心掩码优化提供了一个简单且可迁移的信号。
原文摘要 · Abstract (English)
Object-centric models often produce fragmented masks, boundary leakage, and incorrect region merging. We introduce Similarity-Shift Refinement (SSR), a training-free post-hoc method for improving object-centric masks with a frozen self-supervised Vision Transformer. SSR measures changes in pairwise patch similarity before and after self-attention value aggregation, retains positively strengthened relations, and constructs a sparse affinity graph. This graph propagates the initial soft slot assignments in a single refinement step, without retraining or modifying either model. Across natural-image, synthetic-video, and real-world-video benchmarks, SSR improves all-pixel Adjusted Rand Index in all 24 evaluated model-dataset combinations, with an average gain of 8.5 percentage points. Ablations show that value-space similarity shifts outperform query- and key-space variants as well as static Transformer affinities. However, texture-dense scenes may cause visually similar regions to be over-grouped. Overall, SSR provides a simple and transferable signal for training-free object-centric mask refinement.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。