通过屏蔽视频无关区域，显著降低神经形态芯片的能耗与延迟。
Region Masking to Accelerate Video Processing on Neuromorphic Hardware
- 输入阶段识别关键区域，屏蔽无关事件计算
- 遮蔽60%区域使能时积降低1.65倍，精度下降仅1.09%
- 适合边缘设备上的实时视频检测任务
资源受限设备对片上边缘智能的需求快速增长，推动了降低深度学习模型能耗和延迟的方法研究。脉冲神经网络（SNN）因其基于事件的处理方式有望大幅降低能耗而受到关注。我们指出，尽管SNN中的sigma-delta编码可利用视频帧间的时序冗余，但仍存在大量因无关事件引发的冗余计算。本文提出一种区域遮蔽策略，在SNN输入端识别兴趣区域，从而消除来自非重要区域事件的计算与数据传输。实验表明，该方法不仅显著减少网络整体脉冲活动，还大幅提升吞吐率并降低延迟。在Loihi 2芯片上进行视频目标检测验证，遮蔽约60%输入区域可使能量-延迟积降低1.65倍，仅导致[email protected]下降1.09%。
原文摘要 · Abstract (English)
The rapidly growing demand for on-chip edge intelligence on resource-constrained devices has motivated approaches to reduce energy and latency of deep learning models. Spiking neural networks (SNNs) have gained particular interest due to their promise to reduce energy consumption using event-based processing. We assert that while sigma-delta encoding in SNNs can take advantage of the temporal redundancy across video frames, they still involve a significant amount of redundant computations due to processing insignificant events. In this paper, we propose a region masking strategy that identifies regions of interest at the input of the SNN, thereby eliminating computation and data movement for events arising from unimportant regions. Our approach demonstrates that masking regions at the input not only significantly reduces the overall spiking activity of the network, but also provides significant improvement in throughput and latency. We apply region masking during video object detection on Loihi 2, demonstrating that masking approximately 60% of input regions can reduce energy-delay product by 1.65x over a baseline sigma-delta network, with a degradation in [email protected] by 1.09%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。