跨视角多模态分割，提升第一人称与第三人称视角间物体匹配精度。
Cross-View Multi-Modal Segmentation @ Ego-Exo4D Challenges 2025
- 融合视觉掩码与文本描述，增强物体定位能力。
- 跨视角对齐模块缓解视角差异导致的特征偏差。
- 在Ego-Exo4D大规模数据集上排名第二，适合多视角理解任务。
本文针对Ego-Exo4D Correspondence Challenges 2025中的物体对应任务，提出一种跨视角多模态物体分割方法。给定一个视角（如第一人称）的物体查询，目标是预测另一视角（如第三人称）中的对应物体掩码。为此，我们设计了多模态条件融合模块，通过结合视觉掩码和文本描述作为分割条件，提升定位精度。同时，为应对第一人称与第三人称视角间的视觉域差距，引入跨视角物体对齐模块，强制不同视角间物体层面的一致性，增强模型对视角变化的鲁棒性。所提方法在大规模Ego-Exo4D物体对应基准测试中位列第二。代码将公开于https://github.com/lovelyqian/ObjectRelator。
原文摘要 · Abstract (English)
In this report, we present a cross-view multi-modal object segmentation approach for the object correspondence task in the Ego-Exo4D Correspondence Challenges 2025. Given object queries from one perspective (e.g., ego view), the goal is to predict the corresponding object masks in another perspective (e.g., exo view). To tackle this task, we propose a multimodal condition fusion module that enhances object localization by leveraging both visual masks and textual descriptions as segmentation conditions. Furthermore, to address the visual domain gap between ego and exo views, we introduce a cross-view object alignment module that enforces object-level consistency across perspectives, thereby improving the model's robustness to viewpoint changes. Our proposed method ranked second on the leaderboard of the large-scale Ego-Exo4D object correspondence benchmark. Code will be made available at https://github.com/lovelyqian/ObjectRelator.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。