用脉冲网络提升骨骼动作识别效率,兼顾精度与节能。
Signal-SGN++: Topology-Enhanced Time-Frequency Spiking Graph Network for Skeleton-Based Action Recognition
- 融合拓扑感知与时频脉冲动态,构建新型脉冲图网络。
- 在多个数据集上实现比现有脉冲网络更高的准确率,能耗降低显著。
- 适合低功耗场景下的实时动作识别系统开发。
图卷积网络(GCNs)在建模骨骼拓扑结构方面表现优异,但其密集浮点计算带来高能耗。脉冲神经网络(SNNs)具有事件驱动和稀疏激活特性,能有效降低能耗,却难以捕捉人体运动的时频耦合与拓扑依赖关系。为此,本文提出Signal-SGN++,一种拓扑感知的脉冲图网络框架,结合结构自适应与时频脉冲动力学。该网络采用1D脉冲图卷积(1D-SGC)和频域脉冲卷积(FSC)作为主干,实现时空与频域特征联合提取。其中嵌入拓扑迁移自注意力(TSSA)机制,可自适应地在学习到的骨骼拓扑间传递注意力,提升图级敏感度而不增加计算复杂度。此外,辅助的多尺度小波变换融合(MWTF)分支将脉冲特征分解为多分辨率时频表示,通过拓扑感知时频融合(TATF)单元引入结构先验,确保拓扑一致性。大规模基准测试表明,Signal-SGN++在准确率-能效权衡上优于现有SNN方法,并在显著降低能耗的前提下达到与顶尖GCN相当的性能。
原文摘要 · Abstract (English)
Graph Convolutional Networks (GCNs) demonstrate strong capability in modeling skeletal topology for action recognition, yet their dense floating-point computations incur high energy costs. Spiking Neural Networks (SNNs), characterized by event-driven and sparse activation, offer energy efficiency but remain limited in capturing coupled temporal-frequency and topological dependencies of human motion. To bridge this gap, this article proposes Signal-SGN++, a topology-aware spiking graph framework that integrates structural adaptivity with time-frequency spiking dynamics. The network employs a backbone composed of 1D Spiking Graph Convolution (1D-SGC) and Frequency Spiking Convolution (FSC) for joint spatiotemporal and spectral feature extraction. Within this backbone, a Topology-Shift Self-Attention (TSSA) mechanism is embedded to adaptively route attention across learned skeletal topologies, enhancing graph-level sensitivity without increasing computational complexity. Moreover, an auxiliary Multi-Scale Wavelet Transform Fusion (MWTF) branch decomposes spiking features into multi-resolution temporal-frequency representations, wherein a Topology-Aware Time-Frequency Fusion (TATF) unit incorporates structural priors to preserve topology-consistent spectral fusion. Comprehensive experiments on large-scale benchmarks validate that Signal-SGN++ achieves superior accuracy-efficiency trade-offs, outperforming existing SNN-based methods and achieving competitive results against state-of-the-art GCNs under substantially reduced energy consumption.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。