用分层不对称蒸馏融合事件与图像,提升高速运动下的目标追踪
HAD: Hierarchical Asymmetric Distillation to Bridge Spatio-Temporal Gaps in Event-Based Object Tracking
- 分层对齐策略缓解图像与事件数据的时空差异
- 在MVTec-Event、DSEC等数据集上显著优于现有方法
- 适合高动态范围、快速运动场景的目标追踪任务
RGB相机在高空间分辨率下能捕捉丰富纹理细节,而事件相机则具备出色的时序分辨率和高动态范围(HDR)。结合两者互补优势可显著提升在高速运动、高动态范围环境及动态背景干扰下的目标追踪性能。然而,由于成像机制的根本差异,两类模态间存在显著的时空不对称性,阻碍了有效多模态融合。为此,我们提出层级不对称蒸馏(HAD)框架,显式建模并缓解时空不对称性。具体而言,HAD采用分层对齐策略,在最小化信息损失的同时保持学生网络的计算效率与参数紧凑性。大量实验表明,HAD持续优于当前最优方法,全面的消融研究进一步验证了各设计组件的有效性与必要性。代码即将发布。
原文摘要 · Abstract (English)
RGB cameras excel at capturing rich texture details with high spatial resolution, whereas event cameras offer exceptional temporal resolution and a high dynamic range (HDR). Leveraging their complementary strengths can substantially enhance object tracking under challenging conditions, such as high-speed motion, HDR environments, and dynamic background interference. However, a significant spatio-temporal asymmetry exists between these two modalities due to their fundamentally different imaging mechanisms, hindering effective multi-modal integration. To address this issue, we propose {Hierarchical Asymmetric Distillation} (HAD), a multi-modal knowledge distillation framework that explicitly models and mitigates spatio-temporal asymmetries. Specifically, HAD proposes a hierarchical alignment strategy that minimizes information loss while maintaining the student network's computational efficiency and parameter compactness. Extensive experiments demonstrate that HAD consistently outperforms state-of-the-art methods, and comprehensive ablation studies further validate the effectiveness and necessity of each designed component. The code will be released soon.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。