通过循环一致掩码预测，实现跨视角物体对应关系的精准定位。
Learning Cross-View Object Correspondence via Cycle-Consistent Mask Prediction
- 用条件二值分割生成物体查询掩码，指导目标视频中的物体定位。
- 在Ego-Exo4D和HANDAL-X上达到当前最优性能，显著提升跨视角匹配精度。
- 无需标注即可自监督训练，支持推理时继续优化，适合实际部署场景。
我们研究视频中不同视角间物体级视觉对应关系的建立，重点关注第一人称到第三人称及反向的挑战性场景。提出一种基于条件二值分割的简单而有效框架：将物体查询掩码编码为潜在表示，以引导目标视频中对应物体的定位。为获得鲁棒且视角不变的表示，引入循环一致性训练目标：预测的目标视图掩码被投影回源视图，重建原始查询掩码。这一双向约束提供强自监督信号，无需真实标注，并支持测试时训练（TTT）。在Ego-Exo4D和HANDAL-X基准上的实验表明，该优化目标与TTT策略有效，实现了当前最优性能。代码已开源：https://github.com/shannany0606/CCMP。
原文摘要 · Abstract (English)
We study the task of establishing object-level visual correspondence across different viewpoints in videos, focusing on the challenging egocentric-to-exocentric and exocentric-to-egocentric scenarios. We propose a simple yet effective framework based on conditional binary segmentation, where an object query mask is encoded into a latent representation to guide the localization of the corresponding object in a target video. To encourage robust, view-invariant representations, we introduce a cycle-consistency training objective: the predicted mask in the target view is projected back to the source view to reconstruct the original query mask. This bidirectional constraint provides a strong self-supervisory signal without requiring ground-truth annotations and enables test-time training (TTT) at inference. Experiments on the Ego-Exo4D and HANDAL-X benchmarks demonstrate the effectiveness of our optimization objective and TTT strategy, achieving state-of-the-art performance. The code is available at https://github.com/shannany0606/CCMP.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。