用脉冲神经网络实现低功耗无人机地理定位，兼顾精度与效率。
Efficient Spike-driven Transformer for High-performance Drone-View Geo-Localization
- 设计轻量脉冲变压器骨干，结合脉冲门控机制增强关键特征。
- 引入混合状态空间模块，有效捕捉跨图像的长程依赖关系。
- 仅推理时使用骨干网络，显著降低计算开销，适合边缘部署。
基于人工神经网络（ANN）的传统无人机视角地理定位（DVGL）方法已取得优异性能，但其密集计算导致高功耗。相比之下，脉冲神经网络（SNN）凭借脉冲驱动计算具有天然低功耗优势，但其在DVGL中的潜力尚未充分探索。此外，脉冲计算的固有稀疏性在表示学习中易丢失关键信息，并难以对齐异构视觉数据源的长程依赖。为此，我们提出首个专为DVGL设计的SNN框架SpikeViMFormer：采用轻量脉冲驱动变压器骨干提取粗粒度特征；设计脉冲驱动选择性注意力（SSA）模块，通过脉冲门控机制选择性增强特征并突出判别区域；引入脉冲驱动混合状态空间（SHS）模块，利用混合状态空间学习长程依赖。推理阶段仅使用骨干网络以降低计算成本。为确保骨干有效性，提出新型分层重排序对齐学习（HRAL）策略，通过邻域重排序和跨批次一致性保持来直接优化骨干。实验表明，SpikeViMFormer超越现有SNN模型，在与先进ANN对比下仍具竞争力。代码已开源。
原文摘要 · Abstract (English)
Traditional drone-view geo-localization (DVGL) methods based on artificial neural networks (ANNs) have achieved remarkable performance. However, ANNs rely on dense computation, which results in high power consumption. In contrast, spiking neural networks (SNNs), which benefit from spike-driven computation, inherently provide low power consumption. Regrettably, the potential of SNNs for DVGL has yet to be thoroughly investigated. Meanwhile, the inherent sparsity of spike-driven computation for representation learning scenarios also results in loss of critical information and difficulties in learning long-range dependencies when aligning heterogeneous visual data sources. To address these, we propose SpikeViMFormer, the first SNN framework designed for DVGL. In this framework, a lightweight spike-driven transformer backbone is adopted to extract coarse-grained features. To mitigate the loss of critical information, the spike-driven selective attention (SSA) block is designed, which uses a spike-driven gating mechanism to achieve selective feature enhancement and highlight discriminative regions. Furthermore, a spike-driven hybrid state space (SHS) block is introduced to learn long-range dependencies using a hybrid state space. Moreover, only the backbone is utilized during the inference stage to reduce computational cost. To ensure backbone effectiveness, a novel hierarchical re-ranking alignment learning (HRAL) strategy is proposed. It refines features via neighborhood re-ranking and maintains cross-batch consistency to directly optimize the backbone. Experimental results demonstrate that SpikeViMFormer outperforms state-of-the-art SNNs. Compared with advanced ANNs, it also achieves competitive performance.Our code is available at https://github.com/ISChenawei/SpikeViMFormer
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。