提出简单高效的语义对应框架,低分辨率下仍达顶尖性能。
SimpleMatch: A Simple and Strong Baseline for Semantic Correspondence
- 用轻量上采样解码器恢复深层特征空间细节
- 252x252分辨率下SPair-71k上达84.1% [email protected]
- 适合追求高效实用基准的视觉匹配研究者
近期语义对应方法主要依赖预训练大规模模型,但其对高分辨率输入的依赖导致计算开销大。本文针对现有方法中深层下采样造成邻近关键点特征不可逆融合的问题,提出SimpleMatch框架。该框架通过轻量级上采样解码器将深层特征逐步恢复至1/4分辨率,并引入多尺度监督损失,确保不同尺度下特征具有判别性。同时采用稀疏匹配与窗口定位策略,训练内存减少51%。在252x252分辨率(较当前SOTA缩小3.3倍)下,SPair-71k基准测试取得84.1% [email protected]的优异性能。代码已开源。
原文摘要 · Abstract (English)
Recent advances in semantic correspondence have been largely driven by the use of pre-trained large-scale models. However, a limitation of these approaches is their dependence on high-resolution input images to achieve optimal performance, which results in considerable computational overhead. In this work, we address a fundamental limitation in current methods: the irreversible fusion of adjacent keypoint features caused by deep downsampling operations. This issue is triggered when semantically distinct keypoints fall within the same downsampled receptive field (e.g., 16x16 patches). To address this issue, we present SimpleMatch, a simple yet effective framework for semantic correspondence that delivers strong performance even at low resolutions. We propose a lightweight upsample decoder that progressively recovers spatial detail by upsampling deep features to 1/4 resolution, and a multi-scale supervised loss that ensures the upsampled features retain discriminative features across different spatial scales. In addition, we introduce sparse matching and window-based localization to optimize training memory usage and reduce it by 51%. At a resolution of 252x252 (3.3x smaller than current SOTA methods), SimpleMatch achieves superior performance with 84.1% [email protected] on the SPair-71k benchmark. We believe this framework provides a practical and efficient baseline for future research in semantic correspondence. Code is available at: https://github.com/hailong23-jin/SimpleMatch.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。