通过分层运动估计与内存优化，提升SAM2在长期跟踪中的鲁棒性。
HiM2SAM: Enhancing SAM2 with Hierarchical Motion Estimation and Memory Optimization towards Long-term Tracking
- 分层运动估计：轻量线性预测+选择性非线性修正，无需额外训练。
- 长短期记忆分离：提升遮挡和外观变化下的跟踪稳定性。
- 无需训练、开销低，小模型性能提升更显著，适合部署优化场景。
本文针对视频目标跟踪中的遮挡、背景干扰和目标重现身等问题，对SAM2框架进行改进。提出分层运动估计策略，结合轻量级线性预测与选择性非线性修正，在不需额外训练的前提下提升跟踪精度。同时优化记忆库，区分长短期记忆帧，增强在长时间遮挡和外观变化下的跟踪可靠性。实验表明，不同规模模型均取得一致提升。大模型在LaSOT和LaSOText上分别实现9.6%和7.2%的相对AUC提升，小模型收益更大，验证了该方法在无训练、低开销条件下提升长期跟踪性能的有效性。代码已开源。
原文摘要 · Abstract (English)
This paper presents enhancements to the SAM2 framework for video object tracking task, addressing challenges such as occlusions, background clutter, and target reappearance. We introduce a hierarchical motion estimation strategy, combining lightweight linear prediction with selective non-linear refinement to improve tracking accuracy without requiring additional training. In addition, we optimize the memory bank by distinguishing long-term and short-term memory frames, enabling more reliable tracking under long-term occlusions and appearance changes. Experimental results show consistent improvements across different model scales. Our method achieves state-of-the-art performance on LaSOT and LaSOText with the large model, achieving 9.6% and 7.2% relative improvements in AUC over the original SAM2, and demonstrates even larger relative gains on smaller models, highlighting the effectiveness of our trainless, low-overhead improvements for boosting long-term tracking performance. The code is available at https://github.com/LouisFinner/HiM2SAM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。