提出慢快双轨追踪框架,兼顾高精度与低延迟。
Towards Low-Latency Event Stream-based Visual Object Tracking: A Slow-Fast Approach
- 用图结构学习事件流,分设慢快双分支追踪器。
- 快追踪器单次前传输出多框,延迟低于10ms。
- 适合嵌入式设备实时追踪,尤其低功耗场景。
现有追踪算法多依赖低帧率RGB相机和计算量大的深度神经网络,难以实现低延迟,且在资源受限环境下表现不佳。基于生物启发事件相机的视觉目标追踪近年成为有前景的研究方向,具备低延迟优势。本文提出一种灵活适配不同需求的慢快追踪范式SFTrack:在计算资源充足时采用高精度慢追踪器,在低延迟、资源受限环境中使用高效快追踪器。该框架首先对高时间分辨率事件流进行图表示学习,再将学习到的图结构信息分别融入两个FlashAttention驱动的视觉主干网络,生成慢、快追踪器。快追踪器通过轻量化设计与单次前传输出多个边界框实现低延迟。最后通过监督微调融合双追踪器,并采用知识蒸馏进一步提升快追踪器性能。在FE240、COESOT和EventVOT等多个公开基准上的实验表明,该方法在不同真实场景下兼具有效性与高效性。源代码已开源于https://github.com/Event-AHU/SlowFast_Event_Track。
原文摘要 · Abstract (English)
Existing tracking algorithms typically rely on low-frame-rate RGB cameras coupled with computationally intensive deep neural network architectures to achieve effective tracking. However, such frame-based methods inherently face challenges in achieving low-latency performance and often fail in resource-constrained environments. Visual object tracking using bio-inspired event cameras has emerged as a promising research direction in recent years, offering distinct advantages for low-latency applications. In this paper, we propose a novel Slow-Fast Tracking paradigm that flexibly adapts to different operational requirements, termed SFTrack. The proposed framework supports two complementary modes, i.e., a high-precision slow tracker for scenarios with sufficient computational resources, and an efficient fast tracker tailored for latency-aware, resource-constrained environments. Specifically, our framework first performs graph-based representation learning from high-temporal-resolution event streams, and then integrates the learned graph-structured information into two FlashAttention-based vision backbones, yielding the slow and fast trackers, respectively. The fast tracker achieves low latency through a lightweight network design and by producing multiple bounding box outputs in a single forward pass. Finally, we seamlessly combine both trackers via supervised fine-tuning and further enhance the fast tracker's performance through a knowledge distillation strategy. Extensive experiments on public benchmarks, including FE240, COESOT, and EventVOT, demonstrate the effectiveness and efficiency of our proposed method across different real-world scenarios. The source code has been released on https://github.com/Event-AHU/SlowFast_Event_Track.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。