通过分段对齐增强，减少语音识别模型的删除错误。
SegAug: CTC-Aligned Segmented Augmentation For Robust RNN-Transducer Based Speech Recognition
- 基于对齐生成多样语音文本对，打破模型对连续词序的依赖。
- 在小规模数据上降低12.5%错误率，删除错误减少45.4%。
- 适合提升低资源或跨领域场景下的语音识别鲁棒性。
RNN-Transducer（RNN-T）是语音识别中广泛采用的端到端架构，但其预测器易过度依赖训练数据中的连续词序，导致对罕见或域外短语的删除错误偏高。现有方法如正则化与数据增强常以牺牲其他性能为代价。本文提出SegAug，一种基于对齐的分段增强技术，生成语义层次上差异显著的音频-文本对。该方法促使模型更关注声学特征，同时多样化内部语言模型所学的文本模式，从而降低删除错误并提升整体性能。在LibriSpeech和Tedlium-v3数据集上的评估显示，小规模设置下相对词错误率（WER）降低12.5%，大规模设置下降低6.9%；其中大部分改进源于删除错误的减少，分别达45.4%和18.5%。结果表明，SegAug能有效增强RNN-T的鲁棒性，为复杂多变场景下的语音识别提供有力解决方案。
原文摘要 · Abstract (English)
RNN-Transducer (RNN-T) is a widely adopted architecture in speech recognition, integrating acoustic and language modeling in an end-to-end framework. However, the RNN-T predictor tends to over-rely on consecutive word dependencies in training data, leading to high deletion error rates, particularly with less common or out-of-domain phrases. Existing solutions, such as regularization and data augmentation, often compromise other aspects of performance. We propose SegAug, an alignment-based augmentation technique that generates contextually varied audio-text pairs with low sentence-level semantics. This method encourages the model to focus more on acoustic features while diversifying the learned textual patterns of its internal language model, thereby reducing deletion errors and enhancing overall performance. Evaluations on the LibriSpeech and Tedlium-v3 datasets demonstrate a relative WER reduction of up to 12.5% on small-scale and 6.9% on large-scale settings. Notably, most of the improvement stems from reduced deletion errors, with relative reductions of 45.4% and 18.5%, respectively. These results highlight SegAug's effectiveness in improving RNN-T's robustness, offering a promising solution for enhancing speech recognition performance across diverse and challenging scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。