基于SAM2的多目标跟踪分割框架,提升身份一致性与内存效率。
Seg2Track-SAM2: SAM2-based Multi-object Tracking and Segmentation
- 结合预训练检测器与SAM2,通过专用模块实现初始化、关联与优化。
- KITTI MOTS与MOTS Challenge上排名第四,关联准确率最高。
- 滑动窗口内存策略降低75%内存占用,适合资源受限场景部署。
自动驾驶感知系统需在动态环境中实现鲁棒的多目标跟踪(MOT),以保持物体身份一致性并保障空间精度。近期基础模型如SAM2可无任务特定微调地进行视频分割,但其直接应用于多目标跟踪与分割(MOTS)仍受限于缺乏显式身份管理机制及跟踪过程中的内存增长问题。本文提出Seg2Track-SAM2框架,将预训练检测器与SAM2结合,并引入专用的Seg2Track模块,支持轨迹初始化、数据关联与轨迹精炼。该方法无需数据集特定微调且对检测器无依赖。在KITTI MOTS和MOTS Challenge基准上的实验表明,Seg2Track-SAM2在两个数据集上均位列第四,同时在所有对比方法中取得最高关联准确率(AssA)。此外,滑动窗口内存策略将内存使用减少高达75%,对追踪性能影响极小,显著提升资源受限环境下的部署能力。结果表明,该方法在无需特定训练的前提下,有效提升了MOTS的身份一致性和内存效率。代码已开源:https://github.com/hcmr-lab/Seg2Track-SAM2。
原文摘要 · Abstract (English)
Autonomous-driving perception systems require robust Multi-Object Tracking (MOT) to operate reliably in dynamic environments. MOT maintains consistent object identities across frames while preserving spatial accuracy. Recent foundation models, such as SAM2, provide promptable video segmentation without task-specific fine-tuning. However, their direct application to Multi-Object Tracking and Segmentation (MOTS) remains limited by the absence of explicit identity management mechanisms and by growing memory requirements during tracking. This work introduces Seg2Track-SAM2, a framework that integrates pretrained object detectors with SAM2 and a dedicated Seg2Track module to support track initialization, data association, and track refinement. The method operates without dataset-specific fine-tuning and remains detector-agnostic. Experimental evaluation on the KITTI MOTS and MOTS Challenge benchmarks shows that Seg2Track-SAM2 ranks fourth overall in both datasets while achieving the highest association accuracy (AssA) among compared methods. In addition, a sliding-window memory strategy reduces memory usage by up to 75% with minimal impact on tracking performance, enabling deployment under resource constraints. Together, these results indicate that Seg2Track-SAM2 improves identity consistency and memory efficiency in MOTS without requiring dataset-specific training. The code is available at https://github.com/hcmr-lab/Seg2Track-SAM2.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。