arXiv:2605.22538cs.CV2026-05被引 2

用运动几何语义三重适应,让SAM2更懂复杂目标跟踪

Segment Anything with Motion, Geometry, and Semantic Adaptation for Complex Nonlinear Visual Object Tracking

论文配图:Segment Anything with Motion, Geometry, and Semantic Adaptation for Complex Nonlinear Visual Object Tracking
图 1 · 摘自论文原文
  • 引入轻量非线性运动预测器,动态引导掩码选择与记忆过滤
  • 在多个基准上超越现有SAM2方法,抗无人机干扰能力提升显著
  • 适合需要强泛化、应对遮挡与非线性运动的复杂跟踪场景

传统视觉目标跟踪方法依赖特定任务的监督训练,难以泛化到未见目标及包含干扰物、遮挡和非线性运动的复杂场景。近期视觉基础模型如SAM 2通过大规模预训练学习了强大的视频理解先验,为构建更鲁棒、通用的跟踪器提供了可能。然而直接应用SAM 2于跟踪仍不理想,因其未显式建模目标运动动态,也未强制跨帧的几何与语义一致性,而这二者对可靠跟踪至关重要。为此,我们提出SAMOSA框架,通过显式利用运动、几何和语义线索,将SAM 2适配至复杂视觉目标跟踪场景。具体而言,引入轻量级非线性运动预测器以建模目标动态,并指导掩码选择与记忆过滤;进一步利用语义线索检测目标位移并恢复跟踪失败,几何线索则作为结构约束提升跟踪稳定性。由此,SAMOSA弥合了SAM 2隐式视频理解先验与显式跟踪建模之间的差距。大量实验表明,SAMOSA在通用基准上持续优于现有基于SAM 2的方法,在泛化能力上超越监督式VOT方法,并在抗无人机(anti-UAV)数据集上取得显著提升,该数据集代表复杂非线性运动场景。代码已开源:https://github.com/DurYi/SAMOSA。

原文摘要 · Abstract (English)

Traditional visual object tracking (VOT) methods typically rely on task-specific supervised training, limiting their generalization to unseen objects and challenging scenarios with distractors, occlusion, and nonlinear motion. Recent vision foundation models, exemplified by SAM 2, learn strong video understanding priors from large-scale pretraining and offer a promising foundation for building more robust and generalizable trackers. However, directly applying SAM 2 to VOT remains suboptimal, as it does not explicitly model target motion dynamics or enforce geometric and semantic consistency across frames, both of which are essential for reliable tracking. To address this issue, we propose SAMOSA, a new tracking framework that adapts SAM 2 to complex VOT scenarios by explicitly leveraging motion, geometry, and semantic cues. Specifically, we introduce a lightweight nonlinear motion predictor to model target dynamics and guide mask selection as well as memory filtering. We further exploit semantic cues to detect target shifts and recover from tracking failures, while geometric cues are incorporated as structural constraints to improve tracking stability. In this way, SAMOSA bridges the gap between the implicit video understanding prior of SAM 2 and explicit tracking-oriented modeling. Extensive experiments show that SAMOSA consistently outperforms state-of-the-art SAM 2--based approaches on general benchmarks, demonstrates stronger generalization than supervised VOT methods, and achieves substantial gains on anti-UAV datasets, which typify complex nonlinear motion scenarios. Our code is available at https://github.com/DurYi/SAMOSA.

目标跟踪SAM2非线性运动多模态融合

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。