提出基于时间不确定性评估的脉冲注意力重要性方法,提升神经形态视觉效率。
Uncertainty-Aware Token Importance Estimation in Spiking Transformers

- 用狄利克雷分布建模逐帧类别证据,捕捉脉冲过程中的不确定性变化
- 在多个基准上实现精度与效率平衡,剪枝时增益最显著
- 无需训练、可即插即用,揭示了不确定性与信息贡献的关联
脉冲变压器在神经形态视觉中展现出巨大潜力,但多脉冲步骤中的标记处理仍存在大量冗余和推理开销。现有标记压缩方法主要依赖响应线索,如激活强度、放电统计或特征相似性。尽管有效,这些标准并未从时序演化的类别证据角度显式刻画标记重要性。在脉冲变压器中,标记表示是跨多个脉冲步骤逐步形成的,而非单一时刻确定,因此标记重要性应不仅基于瞬时响应,还应结合时间不确定性模式。我们的核心观察是:标记随时间呈现异构不确定性轨迹,其时间累积不确定性统计可有效区分信息性标记与冗余标记。受此启发,我们提出 Uncert——一种针对脉冲变压器的免训练、即插即用的标记重要性估计框架。具体而言,Uncert 使用狄利克雷分布建模标记级类别证据,并通过脉冲步骤间的均值与波动总结每个标记的时间不确定性,生成用于推理阶段标记压缩的不确定性感知重要性分数。在静态与神经形态基准上的实验表明,Uncert 在准确率与效率之间取得了良好权衡,尤其在标记剪枝场景下表现最稳定。进一步分析揭示了时间不确定性模式与标记贡献之间的明确经验关联,为脉冲变压器中的标记动态提供了新见解。
原文摘要 · Abstract (English)
Spiking transformers have shown strong potential for neuromorphic vision, yet their token processing across multiple spiking steps still introduces substantial redundancy and inference cost. Existing token reduction methods mainly rely on response based cues, such as activation magnitude, firing statistics, or feature similarity. Although effective, these criteria do not explicitly characterize token importance from the perspective of temporally evolving class evidence. In spiking transformers, token representations are progressively formed across multiple spiking steps rather than determined at a single instant, suggesting that token importance should be evaluated not only by instantaneous responses but also by temporal uncertainty patterns. Our key observation is that tokens exhibit heterogeneous uncertainty trajectories over time, and that their temporally aggregated uncertainty statistics provide an effective cue for distinguishing informative tokens from redundant ones. Motivated by this, we propose Uncert, a training free and plug and play token importance estimation framework for spiking transformers. Specifically, Uncert models token wise class evidence with a Dirichlet distribution and summarizes each token temporal uncertainty using its mean and fluctuation across spiking steps, yielding an uncertainty aware importance score for token reduction during inference. Experiments on both static and neuromorphic benchmarks show that Uncert achieves favorable accuracy and efficiency tradeoffs, with the most consistent gains observed under token pruning. Further analysis reveals a clear empirical connection between temporal uncertainty patterns and token contribution, offering new insights into token dynamics in spiking transformers.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。