用生成式语言模型动态构建真实声学场景,提升语音系统抗噪能力。
DGSNA: Dynamic Generative Scene-based Noise Addition method
- 基于提示词的动态生成场景信息,自动构造声学环境参数。
- 结合时频扩散模型合成特定场景噪声,提升语音识别鲁棒性11.32%。
- 无需预存噪声库,适合语音识别与关键词检测模型训练。
为确保语音系统在多样环境下的可靠运行,噪声添加方法已成为标准解决方案。然而,现有方法对真实场景覆盖有限,依赖预存噪声库和场景元数据。本文提出基于提示词的动态生成式场景噪声添加方法(DGSNA),融合动态场景信息生成(DGSI)与基于场景的语音噪声添加(SNAS)。DGSI模块采用背景-示例-任务(BET)提示框架,动态生成符合逻辑的场景信息,包括场景尺寸、声源位置及麦克风布置,解决场景枚举与细节描述难题。SNAS模块则使用基于时频扩散的文生音频模型(TFD)合成特定场景噪声,并通过房间冲激响应(RIR)滤波器将噪声与干净语音融合,简化了传统上繁琐的多场景声学模拟过程。实验表明,DGSNA显著提升语音识别与关键词检测模型的鲁棒性,相对性能提升最高达11.32%。此外,DGSNA与现有噪声添加技术高度兼容。代码与演示已开源:https://dgsna.github.io。
原文摘要 · Abstract (English)
To ensure the reliable operation of speech systems across diverse environments, noise addition methods have emerged as the standard solution.However, existing methods offer limited coverage of real-world scenes and depend on pre-existing noise libraries and scene metadata.This paper presents prompt-based Dynamic Generative Scene-based Noise Addition (DGSNA), a novel approach driven by generative language models that integrates Dynamic Generation of Scene-based Information (DGSI) with Scene-based Noise Addition for Speech (SNAS).The DGSI module, with a BET (Background, Examples, Task) prompt framework, dynamically generates logic-compliant scene-based information, including scene dimensions, sound sources, and microphone positions, thereby addressing the challenges of scene enumeration and detailed description.Complementing this, the SNAS module employs a Time-Frequency Diffusion-based (TFD) Text-to-Audio model to synthesize scene-specific noise. By integrating this noise with clean speech via Room Impulse Response (RIR) filters, the module streamlines the traditionally labor-intensive process of replicating diverse acoustic environments.Experimental results show that DGSNA significantly enhances the robustness of speech recognition and keyword spotting models, achieving relative improvements of up to 11.32\%. Furthermore, DGSNA is highly compatible with existing noise addition techniques. Our implementation and demonstrations are available at https://dgsna.github.io.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。