用轻量MobileViT实现高效多模态跟踪，适合移动端部署。
Lightweight RGB-T Tracking with Mobile Vision Transformers
- 基于MobileViT设计渐进式融合框架，分离建模单模与跨模态交互
- 参数少于400万，CPU上达25.7帧，GPU上达122帧，支持实时运行
- 首个基于MobileViT的多模态跟踪模型，适合嵌入式和移动设备
单模态跟踪（仅RGB）在低光照、恶劣天气和遮挡条件下表现不佳。多模态跟踪通过融合互补信息提升鲁棒性。尽管基于视觉变换器的跟踪器精度高，但通常体积庞大难以实时运行。本文提出一种轻量级RGB-T跟踪器，基于MobileViT并采用渐进式融合框架，利用可分离的混合注意力建模模内与模间交互。该设计生成紧凑且高效的特征表示，实现精确定位，参数量低于400万，在CPU上达到25.7 FPS，GPU上达122 FPS，支持嵌入式与移动平台。据我们所知，这是首个基于MobileViT的多模态跟踪器。代码与权重已开源于GitHub。
原文摘要 · Abstract (English)
Single-modality tracking (RGB-only) struggles under low illumination, weather, and occlusion. Multimodal tracking addresses this by combining complementary cues. While Vision Transformer-based trackers achieve strong accuracy, they are often too large for real-time. We propose a lightweight RGB-T tracker built on MobileViT with a progressive fusion framework that models intra- and inter-modal interactions using separable mixed attention. This design delivers compact, effective features for accurate localization, with under 4M parameters and real-time performance of 25.7 FPS on the CPU and 122 FPS on the GPU, supporting embedded and mobile platforms. To the best of our knowledge, this is the first MobileViT-based multimodal tracker. Model code and weights are available in the GitHub repository.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。