arXiv:2510.18822cs.CV2025-10被引 1

统一视频追踪框架,支持任意粒度目标的精准跟踪。

SAM 2++: Tracking Anything at Any Granularity

  • 设计通用提示编码与统一解码器,实现多粒度输入输出一体化。
  • 提出任务自适应记忆机制,避免不同粒度间参数干扰。
  • 构建首个三粒度标注的大规模视频追踪数据集Tracking-Any-Granularity。

由于不同任务中目标状态粒度差异,现有追踪器多针对单一任务设计,限制了泛化能力,难以有效利用多任务训练数据,造成模型结构与参数冗余。尽管近期统一视觉模型在任务间共享部分架构,但仍保留任务特异性接口,忽视不同粒度间的共性追踪原理,导致真正统一的视频追踪仍存空白。为此,本文提出SAM 2++,一个可处理掩码、边界框和点三种粒度目标的统一框架,通过提示编码、输出解码与记忆表示的一体化设计实现跨粒度统一。首先,设计任务特定提示将多样输入映射为通用提示嵌入,并引入统一解码器以共同输出形式生成结果,无需重设计整体流程。其次,为满足追踪核心操作——记忆匹配,提出任务自适应记忆机制,在统一不同粒度记忆的同时保留其语义差异,防止全参数共享引发干扰。最后,构建首个涵盖三粒度丰富标注的大型视频追踪数据集Tracking-Any-Granularity,基于分阶段人工标注与模型辅助补全的数据引擎完成。大量实验验证,SAM 2++在多种粒度的追踪任务上均达到新最优性能,确立了统一且鲁棒的追踪范式。

原文摘要 · Abstract (English)

Due to the varying granularity of target states across different tasks, most existing trackers are tailored to a single task, which specificity limits their generalization, preventing them from effectively utilizing multi-task training data and leading to redundancy in both model design and parameters. Although recent unified vision models share partial architectures across tasks, they usually retain task-specific interfaces and overlook the common tracking principle behind different granularities, leaving a gap for truly unified video tracking. To unify video tracking tasks, we present SAM 2++, a unified framework that can handle target states at different granularities, including masks, boxes, and points, through an integrated design of prompt encoding, output decoding, and memory representation. First, to handle different target granularities, we design task-specific prompts that map diverse task inputs into general prompt embeddings, together with a Unified Decoder that produces task results in a common output form without redesigning the overall pipeline. Next, to satisfy memory matching, the core operation of tracking, we introduce a task-adaptive memory mechanism that unifies memory across different granularities while preserving their distinct state semantics, preventing full parameter sharing from causing interference across granularities. Finally, we introduce Tracking-Any-Granularity, the first large and diverse video tracking dataset with rich annotations at three granularities. It is constructed through a customized data engine with phased manual annotation and model-assisted completion, providing a comprehensive resource for training, benchmarking, and analyzing unified tracking models. Comprehensive experiments confirm that SAM 2++ sets a new state of the art across diverse tracking tasks at different granularities, establishing a unified and robust tracking framework.

视频追踪统一框架多粒度SAM

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。