提出分块不确定性增强方法,提升语音关键词识别在分布外场景下的泛化能力。
PatchDSU: Uncertainty Modeling for Out of Distribution Generalization in Keyword Spotting
- 将输入切分为局部块,分别进行不确定性采样增强
- 在多个语音数据集上均优于传统方法,尤其在噪声和跨域测试中表现更稳定
- 适合需要鲁棒语音识别的部署场景,如智能设备或嘈杂环境
深度学习模型在许多任务中表现优异,但通常假设训练与测试数据分布一致。然而在真实语音系统中,因环境、录音条件和说话人差异,分布偏移普遍存在。现有领域偏移不确定性(DSU)方法通过基于输入特征统计对每层神经网络输入进行增强,假设特征服从多元高斯分布并用采样特征替换原输入。该方法在计算机视觉中有效,但在语音任务中面临挑战:语音是随时间变化的频谱信号,不能简单视为静态图像,且整体输入易因稀疏性导致特征统计失真。为此,本文提出PatchDSU,将输入分块后独立进行不确定性增强。我们在Google Speech Commands、Librispeech和TED-LIUM数据集上评估了PatchDSU与DSU及其他方法,并在白噪声和MUSAN音乐噪声条件下测试性能。还分析了模型在未训练数据集上的跨域表现。结果表明,多数情况下两者均优于基线,其中PatchDSU在各类场景中表现更一致稳定。
原文摘要 · Abstract (English)
Deep learning models excel at many tasks but rely on the assumption that training and test data follow the same distribution. This assumption often does not hold in real-world speech systems, where distribution shifts are common due to varying environments, recording conditions, and speaker diversity. The method of Domain Shifts with Uncertainty (DSU) augments the input of each neural network layer based on the input feature statistics. It addresses the problem of out-of-domain generalization by assuming feature statistics follow a multivariate Gaussian distribution and substitutes the input with sampled features from this distribution. While effective for computer vision, applying DSU to speech presents challenges due to the nature of the data. Unlike static visual data, speech is a temporal signal commonly represented by a spectrogram - the change of frequency over time. This representation cannot be treated as a simple image, and the resulting sparsity can lead to skewed feature statistics when applied to the entire input. To tackle out-of-distribution issues in keyword spotting, we propose PatchDSU, which extends DSU by splitting the input into patches and independently augmenting each patch. We evaluated PatchDSU and DSU alongside other methods on the Google Speech Commands, Librispeech, and TED-LIUM. Additionally, we evaluated performance under white Gaussian and MUSAN music noise conditions. We also explored out-of-domain generalization by analyzing model performance on datasets they were not trained on. Overall, in most cases, both PatchDSU and DSU outperform other methods. Notably, PatchDSU demonstrates more consistent improvements across the evaluated scenarios compared to other approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。