用神经形态麦克风+图神经网络，在FPGA上实现低延迟关键词识别
End-to-End Keyword Spotting on FPGA Using Graph Neural Networks with a Neuromorphic Auditory Sensor
- 直接处理事件流音频，跳过传统预处理步骤
- 量化后在Google语音命令数据集上达87.43%准确率，端到端延迟<35微秒
- 适合边缘设备实时语音唤醒，功耗仅1.12瓦
随着移动机器人和嵌入式智能的快速发展，边缘平台对高效本地数据处理的需求日益增长。受人类感官系统启发的神经形态传感器生成稀疏的事件基数据，可编码环境变化。本文首次实现了将神经形态听觉传感器（NAS）与图神经网络（GNN）集成于单一FPGA上的端到端关键词检测系统，可实时处理原始音频事件流。该架构跳过传统信号预处理，直接基于事件流进行推理。通过计算近存储的网络结构，系统实现低延迟、低功耗高效推理。实验结果表明，在经过量化后，于经神经形态传感器处理的Google Speech Commands v2数据集上达到87.43%的准确率，端到端延迟低于35微秒，平均功耗为1.12 W。相关数据集、软件模型及硬件模块已开源：https://github.com/vision-agh/NAS-GNN-KWS。
原文摘要 · Abstract (English)
With the rapid growth of mobile robotics and embedded intelligence, there is an increasing demand for efficient on-device data processing on edge platforms. A promising research direction is the use of neuromorphic sensors inspired by human sensory systems, which generate sparse, event-based data encoding changes in the environment. In this work, we present the first end-to-end FPGA implementation of a keyword spotting system that integrates a Neuromorphic Auditory Sensor (NAS) and a graph neural network (GNN) on a single FPGA device, enabling real-time processing of raw audio data. The proposed architecture eliminates conventional signal preprocessing and operates directly on event-based audio streams. Leveraging a compute-near-memory network architecture, the system achieves efficient inference with low latency and low power consumption. Experimental results demonstrate an accuracy of 87.43% after quantization on the Google Speech Commands v2 dataset processed through the neuromorphic sensor, with end-to-end latency below 35 us and average power consumption of 1.12 W. The processed datasets, software models, and hardware modules are available at https://github.com/vision-agh/NAS-GNN-KWS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。