提出可实时学习全部历史信息的多目标跟踪框架,提升长期遮挡下的追踪能力。
FACT: Feature Adaptive Continual-learning Tracker for Multiple Object Tracking
- 设计自适应持续学习模块,在线融合全部历史跟踪信息。
- 在MOT17和MOT20上达到当前最优实时跟踪性能。
- 适合需要长时记忆与快速响应的复杂视频追踪场景。
多目标跟踪(MOT)需在视频序列中识别多个目标并分配对应ID,常面临遮挡问题。现有方法通过在线学习利用外观线索提升适应性,或通过离线学习利用视频时序信息构建长期记忆。然而,多数在线学习方法无法有效利用全部历史信息,难以应对长期遮挡,且难保持实时性;而基于时序信息的离线方法虽能存储长期记忆,却仅依赖局部历史信息进行追踪。为此,本文提出新型MOT框架——特征自适应持续学习跟踪器(FACT),实现实时追踪与特征学习,并可集成于多种先进特征追踪器中。具体地,设计了特征自适应持续学习(FAC)模块,一个可在追踪过程中在线训练的神经网络,用于自适应地学习所有历史信息。此外,还引入两阶段关联模块以适配持续学习追踪。大量实验表明,该方法在MOT17和MOT20基准上达到当前最优在线追踪性能。代码将在接受后公开。
原文摘要 · Abstract (English)
Multiple object tracking (MOT) involves identifying multiple targets and assigning them corresponding IDs within a video sequence, where occlusions are often encountered. Recent methods address occlusions using appearance cues through online learning techniques to improve adaptivity or offline learning techniques to utilize temporal information from videos. However, most existing online learning-based MOT methods are unable to learn from all past tracking information to improve adaptivity on long-term occlusions while maintaining real-time tracking speed. On the other hand, temporal information-based offline learning methods maintain a long-term memory to store past tracking information, but this approach restricts them to use only local past information during tracking. To address these challenges, we propose a new MOT framework called the Feature Adaptive Continual-learning Tracker (FACT), which enables real-time tracking and feature learning for targets by utilizing all past tracking information. We demonstrate that the framework can be integrated with various state-of-the-art feature-based trackers, thereby improving their tracking ability. Specifically, we develop the feature adaptive continual-learning (FAC) module, a neural network that can be trained online to learn features adaptively using all past tracking information during tracking. Moreover, we also introduce a two-stage association module specifically designed for the proposed continual learning-based tracking. Extensive experiment results demonstrate that the proposed method achieves state-of-the-art online tracking performance on MOT17 and MOT20 benchmarks. The code will be released upon acceptance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。