用物理约束神经网络模拟声带振动与发音过程,实现语音信号反演。
Physics-Informed Neural Networks for Speech Production
- 将声带振动与声道声学方程直接作为神经网络约束条件
- 可同时估计气流速率、声带状态和气压,精度达亚毫秒级
- 适用于语音分析与声学建模,尤其适合缺乏标注数据场景
基于声带与声道物理模型的语音生成分析对声带行为研究和语言学研究至关重要。本文提出一种基于物理信息神经网络(PINNs)的语音生成分析方法,直接在声带振动与声道声学的控制方程上训练网络。声带碰撞导致非可微性与梯度消失,挑战传统PINN应用。本文通过引入可微近似函数,成功在PINN框架内分析声带振动。自激振动周期通常未知,本文将其设为可学习参数,实现周期解求解。通过硬约束方式实现声门气流与声道声学的耦合,无需额外损失项。通过正向与逆向分析验证了方法有效性,证明可从语音信号中同步估计声门气流速率、声带振动状态及声门下压力。同一网络架构可应用于正向与逆向分析,展现方法高度通用性。该方法继承了PINN的无网格计算与自然处理非线性的优势,具有广泛应用前景。
原文摘要 · Abstract (English)
The analysis of speech production based on physical models of the vocal folds and vocal tract is essential for studies on vocal-fold behavior and linguistic research. This paper proposes a speech production analysis method using physics-informed neural networks (PINNs). The networks are trained directly on the governing equations of vocal-fold vibration and vocal-tract acoustics. Vocal-fold collisions introduce nondifferentiability and vanishing gradients, challenging phenomena for PINNs. We demonstrate, however, that introducing a differentiable approximation function enables the analysis of vocal-fold vibrations within the PINN framework. The period of self-excited vocal-fold vibration is generally unknown. We show that by treating the period as a learnable network parameter, a periodic solution can be obtained. Furthermore, by implementing the coupling between glottal flow and vocal-tract acoustics as a hard constraint, glottis-tract interaction is achieved without additional loss terms. We confirmed the method's validity through forward and inverse analyses, demonstrating that the glottal flow rate, vocal-fold vibratory state, and subglottal pressure can be simultaneously estimated from speech signals. Notably, the same network architecture can be applied to both forward and inverse analyses, highlighting the versatility of this approach. The proposed method inherits the advantages of PINNs, including mesh-free computation and the natural incorporation of nonlinearities, and thus holds promise for a wide range of applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。