轻量级声源定位模型,适配树莓派等边缘设备实时运行。
CNN-based Robust Sound Source Localization with SRP-PHAT for the Extreme Edge
- 基于SRP-PHAT特征设计轻量CNN架构,降低计算与内存开销。
- 在多种分辨率下,计算量减少10.32%~73.71%,参数量减少59.77%~94.66%。
- 树莓派上实现8.59毫秒/帧延迟,速度比基线快7.26倍。
在噪声与混响环境中,稳健的声源定位正越来越多地采用深度神经网络结合各类声学特征。然而,当前研究主要关注算法精度优化,导致模型庞大,难以部署于边缘设备。而边缘场景如助听器、机器人交互等亟需实时、低资源消耗的声学推理。为此,本文从基于SRP-PHAT特征的稳健卷积神经网络(Cross3D [16])出发,提出适用于极端边缘的高效轻量化架构。针对SRP特征表示与神经网络,分别设计了可扩展的LC-SRP-Edge与Cross3D-Edge算法,以降低硬件开销。LC-SRP-Edge将插值复杂度及片上内存占用减半;在多分辨率情况下,Cross3D-Edge相较基线减少10.32%~73.71%计算量,59.77%~94.66%模型参数。在精度-效率权衡中,最优版本(EM)仅需127.1 MFLOPS计算、3.71 MByte/s带宽与0.821 MByte片上内存,仍保持与先进方法相当的精度。在Raspberry Pi 4B上实现8.59毫秒/帧端到端延迟,较基线快7.26倍。
原文摘要 · Abstract (English)
Robust sound source localization for environments with noise and reverberation are increasingly exploiting deep neural networks fed with various acoustic features. Yet, state-of-the-art research mainly focuses on optimizing algorithmic accuracy, resulting in huge models preventing edge-device deployment. The edge, however, urges for real-time low-footprint acoustic reasoning for applications such as hearing aids and robot interactions. Hence, we set off from a robust CNN-based model using SRP-PHAT features, Cross3D [16], to pursue an efficient yet compact model architecture for the extreme edge. For both the SRP feature representation and neural network, we propose respectively our scalable LC-SRP-Edge and Cross3D-Edge algorithms which are optimized towards lower hardware overhead. LC-SRP-Edge halves the complexity and on-chip memory overhead for the sinc interpolation compared to the original LC-SRP [19]. Over multiple SRP resolution cases, Cross3D-Edge saves 10.32~73.71% computational complexity and 59.77~94.66% neural network weights against the Cross3D baseline. In terms of the accuracy-efficiency tradeoff, the most balanced version (EM) requires only 127.1 MFLOPS computation, 3.71 MByte/s bandwidth, and 0.821 MByte on-chip memory in total, while still retaining competitiveness in state-of-the-art accuracy comparisons. It achieves 8.59 ms/frame end-to-end latency on a Rasberry Pi 4B, which is 7.26x faster than the corresponding baseline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。