轻量级多模态跟踪器，兼顾精度与实时性。
LightFC-X: Lightweight Convolutional Tracker for RGB-X Tracking
- 设计轻量跨模态注意力与时空模板聚合模块
- 参数减少2.6倍，速度提升2.7倍，仍保持领先性能
- 适合在移动端或嵌入式设备上部署
尽管多模态跟踪取得了显著进展，但现有方法对资源受限设备仍过于沉重。为此，我们提出LightFC-X，一种基于统一卷积架构的轻量级多模态跟踪器家族。核心思路是实现轻量级跨模态建模，并联合优化多模态特征与目标时空外观特征。具体提出新型高效跨注意力模块（ECAM），仅用0.08M参数即可完成模板-搜索区域特征的跨模态交互；以及新型时空模板聚合模块（STAM），通过模块微调范式增强模型对时序信息的利用。大量实验表明，LightFC-X达到最优性能、参数量与速度平衡。例如，LightFC-T-ST在LasHeR基准上比CMD在SR和PR上分别高出4.3%和5.7%，参数减少2.6倍，速度提升2.7倍，可在CPU上以22帧/秒实现实时运行。
原文摘要 · Abstract (English)
Despite great progress in multimodal tracking, these trackers remain too heavy and expensive for resource-constrained devices. To alleviate this problem, we propose LightFC-X, a family of lightweight convolutional RGB-X trackers that explores a unified convolutional architecture for lightweight multimodal tracking. Our core idea is to achieve lightweight cross-modal modeling and joint refinement of the multimodal features and the spatiotemporal appearance features of the target. Specifically, we propose a novel efficient cross-attention module (ECAM) and a novel spatiotemporal template aggregation module (STAM). The ECAM achieves lightweight cross-modal interaction of template-search area integrated feature with only 0.08M parameters. The STAM enhances the model's utilization of temporal information through module fine-tuning paradigm. Comprehensive experiments show that our LightFC-X achieves state-of-the-art performance and the optimal balance between parameters, performance, and speed. For example, LightFC-T-ST outperforms CMD by 4.3% and 5.7% in SR and PR on the LasHeR benchmark, which it achieves 2.6x reduction in parameters and 2.7x speedup. It runs in real-time on the CPU at a speed of 22 fps. The code is available at https://github.com/LiYunfengLYF/LightFC-X.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。