用脉冲神经网络实现低功耗语音合成,性能接近传统模型。
Spiking Vocos: An Energy-Efficient Neural Vocoder
- 基于脉冲网络设计新型语音合成模型,降低计算开销。
- 在保持3.74分UTMOS和3.45分PESQ的前提下,能耗仅为14.7%。
- 适合边缘设备部署,特别关注能效的语音生成应用。
尽管神经语音合成器在速度和保真度方面取得显著进展,但其高能耗仍是限制其在计算资源受限的边缘设备上实际部署的关键障碍。脉冲神经网络(SNN)因其事件驱动特性被广泛认为具有高能效,为低资源场景提供了有前景的解决方案。本文提出Spiking Vocos,一种基于高效Vocos框架构建的新型脉冲神经语音合成器,具备超低能耗。为缓解SNN固有的信息瓶颈,我们设计了脉冲ConvNeXt模块以减少乘加操作,并引入幅度捷径路径以保留关键信号动态。为进一步缩小与人工神经网络(ANN)对手的性能差距,我们提出自架构蒸馏策略以有效迁移知识。同时集成轻量级时序移位模块,以极小计算开销增强跨时间维度的信息融合能力。实验表明,本模型在保持与ANN相当性能的同时,仅消耗14.7%的能耗,其UTMOS和PESQ得分分别为3.74和3.45。源代码已开源。
原文摘要 · Abstract (English)
Despite the remarkable progress in the synthesis speed and fidelity of neural vocoders, their high energy consumption remains a critical barrier to practical deployment on computationally restricted edge devices. Spiking Neural Networks (SNNs), widely recognized for their high energy efficiency due to their event-driven nature, offer a promising solution for low-resource scenarios. In this paper, we propose Spiking Vocos, a novel spiking neural vocoder with ultra-low energy consumption, built upon the efficient Vocos framework. To mitigate the inherent information bottleneck in SNNs, we design a Spiking ConvNeXt module to reduce Multiply-Accumulate (MAC) operations and incorporate an amplitude shortcut path to preserve crucial signal dynamics. Furthermore, to bridge the performance gap with its Artificial Neural Network (ANN) counterpart, we introduce a self-architectural distillation strategy to effectively transfer knowledge. A lightweight Temporal Shift Module is also integrated to enhance the model's ability to fuse information across the temporal dimension with negligible computational overhead. Experiments demonstrate that our model achieves performance comparable to its ANN counterpart, with UTMOS and PESQ scores of 3.74 and 3.45 respectively, while consuming only 14.7% of the energy. The source code is available at https://github.com/pymaster17/Spiking-Vocos.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。