融合SAM2与ReferDINO，提升视频目标分割精度
ReferDINO-Plus: 2nd Solution for 4th PVUW MeViS Challenge at CVPR 2025
- 结合SAM2高精度掩码与ReferDINO视觉语言对齐能力
- 60.43的J&F分数，获CVPR 2025 MeViS挑战赛亚军
- 适配单/多目标场景，适合视频编辑与人机交互应用
指称视频目标分割（RVOS）旨在根据文本描述在整段视频中分割目标对象，因其在视频编辑和人机交互中的应用前景而受到计算机视觉领域广泛关注。近期，ReferDINO通过迁移预训练图像模型中的物体级视觉-语言知识，在该任务上展现出良好性能。本文进一步增强其能力，引入SAM2在掩码质量与对象一致性方面的优势。为有效平衡单目标与多目标场景下的性能表现，提出一种条件掩码融合策略，自适应融合ReferDINO与SAM2的掩码结果。所提方案称为ReferDINO-Plus，于MeViS测试集上取得60.43的\(\mathcal{J}\&\mathcal{F}\)分数，位列CVPR 2025 MeViS PVUW挑战赛第二名。代码已公开于：https://github.com/iSEE-Laboratory/ReferDINO-Plus。
原文摘要 · Abstract (English)
Referring Video Object Segmentation (RVOS) aims to segment target objects throughout a video based on a text description. This task has attracted increasing attention in the field of computer vision due to its promising applications in video editing and human-agent interaction. Recently, ReferDINO has demonstrated promising performance in this task by adapting object-level vision-language knowledge from pretrained foundational image models. In this report, we further enhance its capabilities by incorporating the advantages of SAM2 in mask quality and object consistency. In addition, to effectively balance performance between single-object and multi-object scenarios, we introduce a conditional mask fusion strategy that adaptively fuses the masks from ReferDINO and SAM2. Our solution, termed ReferDINO-Plus, achieves 60.43 \(\mathcal{J}\&\mathcal{F}\) on MeViS test set, securing 2nd place in the MeViS PVUW challenge at CVPR 2025. The code is available at: https://github.com/iSEE-Laboratory/ReferDINO-Plus.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。