改进多模态模型在视频对象分割中的表现，提升语言引导的精准度。
The 1st Solution for 7th LSVOS RVOS Track: SaSaSa2VA
- 通过增强采样和选择性平均提升分割精度
- 在7th LSVOS挑战中取得67.45的J&F分数，排名第一
- 适合关注语言引导视频理解的研究者
指代式视频对象分割（RVOS）需要根据自然语言描述分割并追踪视频中的对象，要求对视觉外观和运动信息有精细理解。基于将多模态大语言模型（MLLM）与视频分割模型SAM2结合的Sa2VA，我们识别出两个限制分割性能的关键瓶颈：稀疏帧采样和依赖单一[SEG]标记处理整段视频。为此提出分割增强与选择性平均的Sa2VA（SaSaSa2VA）来解决。在第7届LSVOS挑战（RVOS赛道）中，SaSaSa2VA获得67.45的$/mathcal{J ext{\&}F}$得分，位列第一，领先第二名2.80分。该结果及消融实验表明，高效的分割增强与测试时集成可显著提升基于语言模型的视频分割能力。代码已发布于Sa2VA仓库：https://github.com/bytedance/Sa2VA。
原文摘要 · Abstract (English)
Referring video object segmentation (RVOS) requires segmenting and tracking objects in videos conditioned on natural-language expressions, demanding fine-grained understanding of both appearance and motion. Building on Sa2VA, which couples a Multi-modal Large Language Model (MLLM) with the video segmentation model SAM2, we identify two key bottlenecks that limit segmentation performance: sparse frame sampling and reliance on a single [SEG] token for an entire video. We propose Segmentation Augmented and Selective Averaged Sa2VA (SaSaSa2VA) to address these issues. On the 7th LSVOS Challenge (RVOS track), SaSaSa2VA achieves a $\mathcal{J\&F}$ of 67.45, ranking first and surpassing the runner-up by 2.80 points. This result and ablation studies demonstrate that efficient segmentation augmentation and test-time ensembling substantially enhance grounded MLLMs for RVOS. The code is released in Sa2VA repository: https://github.com/bytedance/Sa2VA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。