无需微调，用大模型+SAM3实现视频目标分割第一名
The 1st Winner for 5th PVUW MeViS-Text Challenge: Strong MLLMs Meet SAM3 for Referring Video Object Segmentation
- 用Gemini拆解语言描述，定位关键帧并生成特征描述
- SAM3生成初始掩码并追踪全视频，最终得分0.909
- Qwen验证行为一致性，提升预测准确性，适合零样本场景
本文介绍我们在第五届PVUW MeViS-Text挑战赛中的获胜方案。该任务要求在以运动为中心的语言表达下进行指代性视频目标分割，模型需同时理解外观、时间行为与物体交互。我们构建了一个完全无需训练的流水线，结合强健的多模态大语言模型与SAM3。方法包含三个阶段：首先，Gemini-3.1 Pro将目标事件分解为实例级定位目标，选择目标最清晰可见的帧，并生成具有区分性的描述；其次，SAM3-agent在选定帧上生成精确种子掩码，官方SAM3追踪器将其传播至整段视频；最后，通过Qwen3.5-Plus与行为级验证进行修正，解决模糊或语义不一致的预测。本方法未进行任务特定微调，在PVUW 2026 MeViS-Text测试集上取得最终分数0.909064和J&F分数0.7897，排名第一。代码已公开于https://github.com/Moujuruo/MeViSv2_Track_Solution_2026。
原文摘要 · Abstract (English)
This report presents our winning solution to the 5th PVUW MeViS-Text Challenge. The track studies referring video object segmentation under motion-centric language expressions, where the model must jointly understand appearance, temporal behavior, and object interactions. To address this problem, we build a fully training-free pipeline that combines strong multimodal large language models with SAM3. Our method contains three stages. First, Gemini-3.1 Pro decomposes each target event into instance-level grounding targets, selects the frame where the target is most clearly visible, and generates a discriminative description. Second, SAM3-agent produces a precise seed mask on the selected frame, and the official SAM3 tracker propagates the mask through the whole video. Third, a refinement stage uses Qwen3.5-Plus and behavior-level verification to correct ambiguous or semantically inconsistent predictions. Without task-specific fine-tuning, our method ranks first on the PVUW 2026 MeViS-Text test set, achieving a Final score of 0.909064 and a J&F score of 0.7897. The code is available at https://github.com/Moujuruo/MeViSv2_Track_Solution_2026.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。