通过渐进对齐机制提升语音识别模型稳定性
Progressive Alignment Objectives for Aligner-Encoder based ASR
- 在中间层引入对齐目标，让模型逐步建立音素对齐
- 长语音上词错误率降低至3.1/5.6（原5.0/7.8）
- 适合追求高精度长语音识别的工程师和研究者
Aligner-Encoders 是一类新兴的端到端语音识别模型，其通过直接从编码器第u位置预测第u个词元，无需解码器注意力或转导器网格，因此编码器需自主学习对齐。但实践中对齐常在深层突然形成，导致长语音训练敏感且不稳定。本文提出 InterAligner，通过引入中间层对齐目标（InterAligner）与中间层 CTC 损失（InterCTC），使对齐能逐层渐进构建，并稳定优化过程。在 17 层 Conformer 与 LibriSpeech 数据集上，仅使用最终对齐的目标时，测试集清音/杂音的词错误率为 5.0/7.8。加入 InterCTC 后降至 3.4/6.0，再采用 InterAligner 进一步降至 3.1/5.6，尤其在长语音上提升显著。
原文摘要 · Abstract (English)
Aligner-Encoders are recently proposed seq2seq end-to-end ASR models that replace decoder attention by predicting the uth token directly from the u-th encoder position, so the encoder must learn the alignment internally without cross-attention or a transducer lattice. In practice, this alignment often forms abruptly in the upper layers, making training sensitive and brittle on long utterances. We propose InterAligner, which adds an intermediate Aligner objective so alignment can form progressively across depth, together with an intermediate CTC loss (InterCTC) to stabilize optimization. On LibriSpeech with a 17-layer Conformer, a final-only Aligner reaches 5.0/7.8 WER (test-clean/other). InterCTC improves to 3.4/6.0, and InterAligner further reduces WER to 3.1/5.6 with the largest gains on long utterances.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。