通过扩大视频帧与分割标记，提升语言-视频理解精度。
SVAC: Scaling Is All You Need For Referring Video Object Segmentation
- 扩大输入帧和分割令牌，增强视频与语言的交互
- 在多个基准上达到领先性能，计算效率良好
- 适合需要高精度视频对象分割的研究者使用
指代视频对象分割（RVOS）旨在根据自然语言描述对视频序列中的目标对象进行分割。尽管多模态大语言模型（MLLMs）通过增强文本-视频理解提升了RVOS性能，但仍面临诸多挑战：未能充分挖掘MLLM的先验知识、长时视频带来高昂的计算与内存开销、难以处理复杂的时序动态。本文提出SVAC，一种统一模型，通过扩大输入帧数与分割令牌数量来增强视频-语言交互和分割精度。为应对由此带来的计算挑战，SVAC引入基于锚点的时空压缩（ASTC）模块，在保留关键时空结构的同时压缩视觉令牌。此外，提出片段特定分配（CSA）策略，更好捕捉视频片段间的动态物体行为。实验表明，SVAC在多个RVOS基准上取得当前最优性能，且具备良好的效率。代码已开源。
原文摘要 · Abstract (English)
Referring Video Object Segmentation (RVOS) aims to segment target objects in video sequences based on natural language descriptions. While recent advances in Multi-modal Large Language Models (MLLMs) have improved RVOS performance through enhanced text-video understanding, several challenges remain, including insufficient exploitation of MLLMs' prior knowledge, prohibitive computational and memory costs for long-duration videos, and inadequate handling of complex temporal dynamics. In this work, we propose SVAC, a unified model that improves RVOS by scaling up input frames and segmentation tokens to enhance video-language interaction and segmentation precision. To address the resulting computational challenges, SVAC incorporates the Anchor-Based Spatio-Temporal Compression (ASTC) module to compress visual tokens while preserving essential spatio-temporal structure. Moreover, the Clip-Specific Allocation (CSA) strategy is introduced to better handle dynamic object behaviors across video clips. Experimental results demonstrate that SVAC achieves state-of-the-art performance on multiple RVOS benchmarks with competitive efficiency. Our code is available at https://github.com/lizhang1998/SVAC.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。