arXiv:2510.07319cs.CV2025-10被引 1

用时间提示提升视频目标分割精度,无需密集标注。

Temporal Prompting Matters: Rethinking Referring Video Object Segmentation

  • 分解任务为指代、视频和分割三部分,利用现成检测器生成时间提示。
  • 通过提示偏好学习评估提示质量,显著提升分割准确性。
  • 适配现有图像分割模型,高效实现视频目标分割,适合资源受限场景。

指代性视频目标分割(RVOS)旨在根据查询句子在视频中分割出对应对象。现有方法多需端到端训练与密集掩码标注,计算成本高且难以扩展。本文重新审视该任务,将RVOS分解为指代、视频和分割三因素,提出时间提示生成与选择框架Tenet,用于处理指代与视频因素,而将分割任务交由基础分割模型完成。为高效适配基于图像的基础分割模型,我们利用现成的目标检测器与追踪器生成与查询句相关的时间提示。尽管高质量时间提示可生成,但其质量难以通过置信度分数识别。为此,我们提出提示偏好学习以评估提示质量。将此类提示输入图像基础分割模型后,可生成高质量掩码,实现对指代性视频目标分割的高效适配。在多个RVOS基准上的实验验证了该框架的有效性。

原文摘要 · Abstract (English)

Referring Video Object Segmentation (RVOS) aims to segment the object referred to by the query sentence in the video. Most existing methods require end-to-end training with dense mask annotations, which could be computation-consuming and less scalable. In this work, we rethink the RVOS problem and aim to investigate the key to this task. Based on existing foundation segmentation models, we decompose the RVOS task into referring, video, and segmentation factors, and propose a Temporal Prompt Generation and Selection (Tenet) framework to address the referring and video factors while leaving the segmentation problem to foundation models. To efficiently adapt image-based foundation segmentation models to referring video object segmentation, we leverage off-the-shelf object detectors and trackers to produce temporal prompts associated with the referring sentence. While high-quality temporal prompts could be produced, they can not be easily identified from confidence scores. To tackle this issue, we propose Prompt Preference Learning to evaluate the quality of the produced temporal prompts. By taking such prompts to instruct image-based foundation segmentation models, we would be able to produce high-quality masks for the referred object, enabling efficient model adaptation to referring video object segmentation. Experiments on RVOS benchmarks demonstrate the effectiveness of the Tenet framework.

视频分割提示学习目标追踪

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。