MARCO让对应点识别更准更通用，小模型也能快速推理。
MARCO: Navigating the Unseen Space of Semantic Correspondence
- 用粗到细目标+自蒸馏框架提升定位精度与泛化能力
- 在细粒度匹配上提升8.9%，未见关键点准确率高5.1%
- 模型小3倍、快10倍，适合实际应用
近期语义对应方法依赖双编码器架构，结合DINOv2与扩散模型，虽精度高但泛化能力差，难以处理训练中未见的关键点。本文基于DINOv2提出MARCO，通过新颖的训练框架实现可泛化的对应关系建模。该方法融合粗到细目标以提升空间精度，并采用自蒸馏机制将稀疏标注扩展至未标注区域，使少量关键点生成稠密且语义一致的对应关系。MARCO在SPair-71k、AP-10K和PF-PASCAL上达到新基准，细粒度定位性能提升8.9%（[email protected]），对未见关键点的泛化能力提升5.1%（SPair-U），跨类别泛化提升4.7%（MP-100）。模型规模仅为扩散方法的1/3，推理速度提升10倍。代码已开源。
原文摘要 · Abstract (English)
Recent advances in semantic correspondence rely on dual-encoder architectures, combining DINOv2 with diffusion backbones. While accurate, these billion-parameter models generalize poorly beyond training keypoints, revealing a gap between benchmark performance and real-world usability, where queried points rarely match those seen during training. Building upon DINOv2, we introduce MARCO, a unified model for generalizable correspondence driven by a novel training framework that enhances both fine-grained localization and semantic generalization. By coupling a coarse-to-fine objective that refines spatial precision with a self-distillation framework, which expands sparse supervision beyond annotated regions, our approach transforms a handful of keypoints into dense, semantically coherent correspondences. MARCO sets a new state of the art on SPair-71k, AP-10K, and PF-PASCAL, with gains that amplify at fine-grained localization thresholds (+8.9 [email protected]), strongest generalization to unseen keypoints (+5.1, SPair-U) and categories (+4.7, MP-100), while remaining 3x smaller and 10x faster than diffusion-based approaches. Code is available at https://github.com/visinf/MARCO .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。