让SAM2零样本追踪更准更快,靠的是会动的内存选择。
SAMURAI: Adapting Segment Anything Model for Zero-Shot Visual Tracking with Motion-Aware Memory
- 用运动信息筛选记忆,避免错误累积
- 在LaSOT上成功率达7.1%提升,GOT-10k精度增3.5%
- 无需微调,适合真实动态场景应用
Segment Anything Model 2 (SAM 2) 在图像分割任务中表现优异,但在视觉目标跟踪方面仍面临挑战,尤其在密集场景、快速移动或自遮挡物体情况下。原始模型采用固定窗口记忆机制,未考虑记忆质量,导致视频中误差传播。本文提出SAMURAI,一种专为视觉目标跟踪优化的SAM 2增强版本。通过引入时间运动线索与提出的运动感知记忆选择机制,SAMURAI能有效预测物体运动并优化掩码选择,实现无需重训练或微调的鲁棒、高精度跟踪。该方法实时运行,在多个基准数据集上展现强大零样本性能。实验表明,其在LaSOT$_{\text{ext}}$上成功率提升7.1% AUC,GOT-10k上平均重叠率(AO)提升3.5%。同时在LaSOT上达到与全监督方法相当的结果,证明其在复杂跟踪场景中的鲁棒性及在动态环境中的实际应用潜力。
原文摘要 · Abstract (English)
The Segment Anything Model 2 (SAM 2) has demonstrated strong performance in object segmentation tasks but faces challenges in visual object tracking, particularly when managing crowded scenes with fast-moving or self-occluding objects. Furthermore, the fixed-window memory approach in the original model does not consider the quality of memories selected to condition the image features for the next frame, leading to error propagation in videos. This paper introduces SAMURAI, an enhanced adaptation of SAM 2 specifically designed for visual object tracking. By incorporating temporal motion cues with the proposed motion-aware memory selection mechanism, SAMURAI effectively predicts object motion and refines mask selection, achieving robust, accurate tracking without the need for retraining or fine-tuning. SAMURAI operates in real-time and demonstrates strong zero-shot performance across diverse benchmark datasets, showcasing its ability to generalize without fine-tuning. In evaluations, SAMURAI achieves significant improvements in success rate and precision over existing trackers, with a 7.1% AUC gain on LaSOT$_{\text{ext}}$ and a 3.5% AO gain on GOT-10k. Moreover, it achieves competitive results compared to fully supervised methods on LaSOT, underscoring its robustness in complex tracking scenarios and its potential for real-world applications in dynamic environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。