arXiv:2412.00877cs.SDeess.AS2024-12

根据样本复杂度动态调整数据增强和损失计算,提升低资源语音识别效果。

Complexity boosted adaptive training for better low resource ASR performance

  • 基于样本复杂度动态调整数据增强强度和中间层损失传播。
  • 在LibriSpeech 100h上相对降低13.4%和14.1%的词错误率(WER)。
  • 适合低资源场景下的语音识别模型优化,尤其对Conformer架构有效。

ASR模型训练过程中,数据增强强度与损失计算通常按预设参数进行。例如,SpecAugment采用固定强度掩码时频域谱图;CTC-based多层模型一般基于编码器最后一层输出计算损失。然而忽略样本动态特性可能导致次优训练。为此,本文提出两阶段复杂度增强自适应(CBA)训练方法:第一阶段使用中间层CTC正则化和非自适应数据增强训练模型;第二阶段引入新颖的自适应策略MinMax-IBF,动态计算样本复杂度,并据此调整数据增强与中间层损失正则化。该方法在Wenet的Conformer架构上实现显著提升:在LibriSpeech 100h测试集clean和other上相对降低13.4%和14.1%的词错误率(WER),在AISHELL-1测试集上相对降低6.3%。

原文摘要 · Abstract (English)

During the entire training process of the ASR model, the intensity of data augmentation and the approach of calculating training loss are applied in a regulated manner based on preset parameters. For example, SpecAugment employs a predefined strength of augmentation to mask parts of the time-frequency domain spectrum. Similarly, in CTC-based multi-layer models, the loss is generally determined based on the output of the encoder's final layer during the training process. However, ignoring dynamic characteristics may suboptimally train models. To address the issue, we present a two-stage training method, known as complexity-boosted adaptive (CBA) training. It involves making dynamic adjustments to data augmentation strategies and CTC loss propagation based on the complexity of the training samples. In the first stage, we train the model with intermediate-CTC-based regularization and data augmentation without any adaptive policy. In the second stage, we propose a novel adaptive policy, called MinMax-IBF, which calculates the complexity of samples. We combine the MinMax-IBF policy to data augmentation and intermediate CTC loss regularization to continue training. The proposed CBA training approach shows considerable improvements, up to 13.4% and 14.1% relative reduction in WER on the LibriSpeech 100h test-clean and test-other dataset and also up to 6.3% relative reduction on AISHELL-1 test set, over the Conformer architecture in Wenet.

语音识别自适应训练低资源

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。