利用耳蜗植入器麦克风的空间线索,提升真实场景下的语音分离效果
Leveraging Spatial Cues from Cochlear Implant Microphones to Efficiently Enhance Speech Separation in Real-World Listening Scenes
- 结合隐式与显式空间线索增强语音分离性能
- 在声音相似或空间相近时,分离准确率显著提升
- 适合耳蜗植入等助听设备在真实环境中的应用
单通道干语音混合的语音分离方法已取得显著进展,但真实世界的声学环境(如空间分布和混响)仍具挑战性,限制了其在耳蜗植入器(CIs)等助听设备中的实际效果。本文量化了真实声学场景对语音分离的影响,并探索空间线索如何高效提升分离质量。分析基于隐式空间线索(模型从输入中学习)和显式空间线索(人工计算并作为辅助输入)。结果表明,两类空间线索均能提升空间分离或邻近说话者混合语句的分离效果;当频谱线索模糊(如声音相似)时,空间线索尤为有效。尤其在单侧耳蜗植入器(单一麦克风)提供的隐式线索较弱时,显式线索仍可显著改善性能。研究强调需在真实世界数据上训练模型以提升泛化能力。统计分析揭示了数据特性对模型表现的影响,为耳蜗植入器及其他助听设备在现实场景中的高效语音分离提供了依据。
原文摘要 · Abstract (English)
Speech separation approaches for single-channel, dry speech mixtures have significantly improved. However, real-world spatial and reverberant acoustic environments remain challenging, limiting the effectiveness of these approaches for assistive hearing devices like cochlear implants (CIs). To address this, we quantify the impact of real-world acoustic scenes on speech separation and explore how spatial cues can enhance separation quality efficiently. We analyze performance based on implicit spatial cues (inherent in the acoustic input and learned by the model) and explicit spatial cues (manually calculated spatial features added as auxiliary inputs). Our findings show that spatial cues (both implicit and explicit) improve separation for mixtures with spatially separated and nearby talkers. Furthermore, spatial cues enhance separation when spectral cues are ambiguous, such as when voices are similar. Explicit spatial cues are particularly beneficial when implicit spatial cues are weak. For instance, single CI microphone recordings provide weaker implicit spatial cues than bilateral CIs, but even single CIs benefit from explicit cues. These results emphasize the importance of training models on real-world data to improve generalizability in everyday listening scenarios. Additionally, our statistical analyses offer insights into how data properties influence model performance, supporting the development of efficient speech separation approaches for CIs and other assistive devices in real-world settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。