首个基于RGB的全脉冲神经网络无人机跟踪框架,低功耗高效定位目标。
STATrack: A Target-Aware Fully Spiking Neural Network for Efficient UAV Tracking
- 用脉冲神经网络处理RGB图像,无需昂贵事件相机。
- 引入自适应互信息最大化,提升目标特征区分度,精度领先。
- 适合资源受限的无人机平台,尤其关注能效与实时性应用。
脉冲神经网络(SNN)以其事件驱动计算和低功耗特性,在无人机(UAV)视觉跟踪中展现出巨大潜力。然而,现有SNN跟踪器多依赖昂贵的事件相机,限制了其在标准RGB相机无人机平台上的部署。为此,我们提出STATrack,一个仅使用RGB输入的全脉冲神经网络框架,据我们所知,这是首个探索全脉冲网络用于RGB基无人机跟踪的研究。针对深度脉冲表示中有限时间步下细粒度目标信息易丢失的问题,我们引入自适应互信息最大化(AMIM),通过最大化目标中心模板与其深层脉冲表示间的统计相关性,促使脉冲主干网络学习更优的目标区分特征。此外,设计了一种样本难度感知的动态加权策略,根据每批训练样本的相对定位难度自适应调整互信息约束。在四个主流无人机跟踪基准上的大量实验表明,STATrack实现了顶尖跟踪性能,且理论能耗极低,凸显其在能源受限无人机应用中的潜力。代码已公开:https://anonymous.4open.science/r/STATrack。
原文摘要 · Abstract (English)
Spiking Neural Networks (SNNs), characterized by their event-driven computation and low power consumption, have shown great potential for energy-efficient visual tracking on unmanned aerial vehicles (UAVs). However, existing SNN-based trackers often rely on costly event cameras, which limits their deployment on standard RGB-camera UAV platforms. To address this limitation, we propose STATrack, a fully spiking neural network framework for UAV visual tracking using only RGB inputs. To the best of our knowledge, this is the first study to explore fully spiking neural networks for RGB-based UAV visual tracking. Considering the challenge of preserving fine-grained target information in deep spiking representations under a limited number of time steps, we introduce Adaptive Mutual Information Maximization (AMIM). AMIM maximizes the statistical dependency between target-centered templates and their deep spiking representations, encouraging the spiking backbone to learn representations that better distinguish the target from surrounding backgrounds. We further develop a sample-difficulty-aware dynamic weighting strategy that adaptively adjusts the mutual-information constraint according to the relative localization difficulty of each training batch. Extensive experiments on four widely used UAV tracking benchmarks demonstrate that STATrack achieves state-of-the-art tracking performance with low theoretical energy consumption, highlighting its potential for energy-constrained UAV applications. Code is released at: https://anonymous.4open.science/r/STATrack.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。