arXiv:2605.28387cs.LGcs.AI2026-05被引 1

在神经形态芯片上实现事件相机的持续动作识别,低功耗高实时。

CLANE: Continual Learning of Actions on Neuromorphic Hardware from Event Cameras

论文配图:CLANE: Continual Learning of Actions on Neuromorphic Hardware from Event Cameras
图 1 · 摘自论文原文
  • 用脉冲2D CNN与可在线学习的脉冲网络提取时空特征
  • 在真实场景数据集上达到70.4%准确率,能持续学习新动作不遗忘
  • 适合需要隐私保护和低延迟的AR/VR与机器人应用

在新兴的AR/VR与机器人应用中,识别并持续学习新的人类动作而不遗忘旧类别至关重要。此类应用要求设备端的处理与学习能力,以保障隐私并实现低延迟自适应。事件相机通过稀疏、异步输出实现高效视觉感知,天然适配神经形态计算。然而,此前尚无系统在神经形态硬件上部署事件驱动的动作识别持续学习流程。本文提出CLANE(Continual Learning of Actions on Neuromorphic Hardware from Event Cameras),全栈部署于Intel Loihi 2芯片。CLANE结合脉冲2D CNN进行时空特征提取,采用扩展后的CLP-SNN作为片上学习头,并引入新颖的时序聚合层与定点归一化层,均专为Loihi 2设计。在真实世界条件下采集的50类动作数据集THU E-ACT-50上,CLANE在持续学习任务中实现70.4%准确率,相较基于边缘GPU的顺序CNN+GRU+CLP基线,能耗降低超过100倍,延迟降低16倍,经三重评估层级的同算法跨平台基准测试验证。

原文摘要 · Abstract (English)

Recognizing and continuously learning novel human actions without forgetting prior classes is a requirement for emerging AR/VR and robotics applications. For these applications, both on-device processing and learning are essential for privacy and low-latency adaptation. Event cameras address the efficiency of visual sensing with sparse, asynchronous output that is naturally compatible with neuromorphic processing. Yet no prior system has deployed a continual on-device learning pipeline for event-based action recognition using neuromorphic hardware. We present CLANE, Continual Learning of Actions on Neuromorphic Hardware from Event Cameras, deployed end-to-end on Intel Loihi 2. CLANE combines a spiking 2D CNN for spatiotemporal feature extraction with CLP-SNN as its on-chip learning head, extended to action clips via a Temporal Aggregation Layer and a fixed-point Normalization Layer, both novel Loihi 2 modules. On THU E-ACT-50, a 50-class dataset captured under real-world conditions, CLANE achieves 70.4% accuracy in a continual learning task while delivering more than 100x energy reduction and 16x lower latency over a sequential CNN+GRU+CLP edge GPU baseline, validated through iso-algorithm cross-platform benchmarking across three evaluation levels.

神经形态计算持续学习事件相机低功耗

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。