用深度学习将环形麦克风阵列转为高质量多说话人空间音频。
Neural Ambisonic Encoding For Multi-Speaker Scenarios Using A Circular Microphone Array
- 两阶段网络架构处理环形阵列信号，输出二阶全向声场数据。
- 新损失函数提升声道间相关性，定位准确率显著提高。
- 适合虚拟现实、远程会议等需精准空间音效的场景。
空间音频格式如全向声学（Ambisonics）具备与播放设备无关的特性，适用于远程会议和虚拟现实等场景。传统全向声学编码依赖球形麦克风阵列以高效捕捉声场，但在实际应用中灵活性受限。本文提出一种基于深度学习的方法，利用两阶段网络架构，将环形麦克风阵列信号编码为二阶全向声学（SOA）数据，适用于多说话人环境。同时引入两项创新：(i) 基于空间功率图的新型损失函数，用于约束全向声学信号的声道间相关性；(ii) 通道重排技术，解决水平环形阵列难以编码垂直信息的歧义问题。在模拟语音与噪声数据集上的评估表明，该方法持续优于传统信号处理（SP）与现有深度学习方法，在音色还原与空间保真度、说话人定位准确性方面均有显著提升。可听化演示与可视化结果见 https://bridgoon97.github.io/NeuralAmbisonicEncoding/。
原文摘要 · Abstract (English)
Spatial audio formats like Ambisonics are playback device layout-agnostic and well-suited for applications such as teleconferencing and virtual reality. Conventional Ambisonic encoding methods often rely on spherical microphone arrays for efficient sound field capture, which limits their flexibility in practical scenarios. We propose a deep learning (DL)-based approach, leveraging a two-stage network architecture for encoding circular microphone array signals into second-order Ambisonics (SOA) in multi-speaker environments. In addition, we introduce: (i) a novel loss function based on spatial power maps to regularize inter-channel correlations of the Ambisonic signals, and (ii) a channel permutation technique to resolve the ambiguity of encoding vertical information using a horizontal circular array. Evaluation on simulated speech and noise datasets shows that our approach consistently outperforms traditional signal processing (SP) and DL-based methods, providing significantly better timbral and spatial quality and higher source localization accuracy. Binaural audio demos with visualizations are available at https://bridgoon97.github.io/NeuralAmbisonicEncoding/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。