CTC模型内部评分失效，需引入语言模型才能提升识别准确率
The Anatomy of the CTC Oracle Gap: Acoustic Exhaustion and Linguistic Recovery
- 发现CTC评分在候选词排序中因空白路径泛滥而失去区分能力
- 引入RoBERTa语言模型后，测试集错误率降低0.535个百分点（显著）
- 该方法无需调参，适用于多种模型和噪声环境，适合语音识别优化者
我们研究了CTC内部评分在N-best候选选择中的局限性，定位到声学置信度与语言合理性之间的信息瓶颈。在LibriSpeech dev-other上，11种CTC内部及声学特征评分策略在G=16时均未显著优于贪婪解码（所有p > 0.05）。声学评分与每句错误率的相关性系统性下降：从G=4时的Spearman ρ=-0.574降至G=128时的-0.270，损失达53%，主因是空白路径泛滥。这表明CTC内部表示的判别能力已饱和，无法通过重组声学信号弥补其与理想解之间的差距。确认瓶颈为语言层面而非声学后，采用最小贝叶斯风险（MBR）解码并引入RoBERTa伪对数似然后验（τ=10, G=128），在测试集上实现5.42%的错误率（贪婪解为5.96%，Δ=-0.535百分点，p<0.0001，相对降低9.0%）。该方法在相同范围内相关性仅下降21%，保持判别力。该方案在两个Zipformer架构、三个领域（LibriSpeech、TED-LIUM 3、VoxPopuli）及四类噪声水平下，13组条件中有11组取得显著增益。训练端，基于CTC前向-后向算法的标准MWER训练实现了输出投影上的Rao-Blackwellized REINFORCE，方差约为Viterbi方法的三分之一；然而在接近收敛的检查点上序列级微调失败，所有四种CR-CTC配置均崩溃（错误率上升6.18至8.90百分点），因训练目标函数的奖励信号仅0.007百分点，无法提供有效梯度。
原文摘要 · Abstract (English)
We study the limits of CTC-internal scoring for N-best hypothesis selection and locate the information bottleneck separating acoustic confidence from linguistic plausibility. Eleven CTC-internal and acoustic-feature scoring strategies produce no statistically significant WER improvement over greedy decoding on LibriSpeech dev-other at G=16 (all p > 0.05). The exhaustion is systematic: CTC's Spearman $ρ$ between hypothesis score and per-utterance WER degrades from -0.574 at G=4 to -0.270 at G=128, a 53% loss driven by blank-path proliferation. This establishes that the discriminative capacity of CTC-internal representations is saturated: no recombination of acoustic signals can close the oracle gap. Confirming that the bottleneck is linguistic, not acoustic, external linguistic information introduced via MBR decoding breaks through it. MBR-CER decoding with a RoBERTa pseudo-log-likelihood (PLL) posterior ($τ$=10, G=128) achieves 5.42% WER on held-out LibriSpeech test-other (greedy 5.96%, $Δ$=-0.535 pp, p<0.0001, 9.0% relative). RoBERTa PLL $ρ$ degrades only 21% over the same range, retaining discriminating power where CTC loses it. Applied without retuning across two Zipformer architectures, three domains (LibriSpeech, TED-LIUM 3, VoxPopuli), and four MUSAN noise levels, the recipe gives significant gains in 11 of 13 conditions. On the training side, standard MWER training via the CTC forward-backward algorithm implements Rao-Blackwellized REINFORCE at the output projection (variance about 3x below Viterbi). Yet sequence-level fine-tuning fails at near-converged checkpoints: all four MWER configurations on CR-CTC collapse (+6.18 to +8.90 pp WER), as a training oracle gap of 0.007 pp provides no usable reward signal.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。