通过双对齐蒸馏压缩多模态追踪模型,实现高速高精度推理。
Towards Compact Unified Multimodal Tracking: Synergizing Knowledge Distillation with Structural Pruning

- 拆解知识迁移为空间与语义双通道,分别优化定位与分类能力。
- 在RGBT234上达91.5% MPR,单卡4090运行达54 FPS,速度提升5倍。
- 适合边缘部署的轻量级多模态追踪场景,尤其关注实时性与效率。
统一多模态目标追踪通过融合RGB、热成像、深度等互补传感器数据实现了显著鲁棒性,但现有模型计算开销大,难以部署于资源受限的边缘设备。本文发现预测头是关键但常被忽视的效率瓶颈。通过优化解码器结构,我们实现实时推理,但引入了轻量学生模型与重型教师模型之间的容量差距。为此,我们系统分析了17种蒸馏策略,提出双对齐蒸馏框架。核心思想是将知识迁移解耦为两个互补流:(1) 空间表征对齐,利用特征蒸馏强化学生模型对前景目标的空间聚焦(“何处追踪”);(2) 语义分布对齐,通过基于logit的蒸馏对齐决策边界,传递判别性暗知识(“追踪什么”)。在五个基准上的实验表明,该方法显著优于复杂先进的方法。值得注意的是,所提蒸馏模型在RGBT234上达到91.5% MPR,单卡RTX 4090下运行达54 FPS,较教师模型提速5倍,且精度更优。
原文摘要 · Abstract (English)
Unified multimodal object tracking has achieved remarkable robustness by leveraging complementary sensor data (e.g., RGB, Thermal, Depth), yet the heavy computational burden of state-of-the-art models hinders their deployment on resource-constrained edge devices. In this work, we identify the prediction head as a critical but often overlooked efficiency bottleneck. By strategically streamlining the decoder architecture, we unlock the potential for real-time inference but simultaneously introduce a capacity gap between the lightweight student and the heavy teacher. To resolve this, we conduct a systematic analysis of 17 distillation strategies and introduce a Dual-Alignment Distillation framework. Our key insight is that effective compression requires decoupling knowledge transfer into two complementary streams: (1) Spatial Representation Alignment, which employs feature distillation to sharpen the student's spatial focus on foreground targets ("Where to track"); and (2) Semantic Distribution Alignment, which utilizes logit-based distillation to align decision boundaries and transfer discriminative dark knowledge ("What to track"). Extensive experiments across five benchmarks demonstrate that our approach significantly outperforms complex state-of-the-art methods. Notably, our distilled model achieves 91.5% MPR on RGBT234 and operates at 54 FPS on a single RTX 4090, representing a 5x speedup over the teacher model while maintaining superior accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。