用大模型生成伪标签，零监督训练流式语音识别模型
Fast Streaming Transducer ASR Prototyping via Knowledge Distillation with Whisper
- 用大模型生成伪标签，实现无监督端到端训练
- 在6种语言上验证有效，即使伪标签噪声大也能工作
- 适合资源有限者快速构建低延迟语音识别系统
无监督或少监督的自动语音识别（ASR）训练仍是开放问题。本文展示，通过使用基础语音模型（FSM）生成的伪标签（PL），可在消费级GPU上从头训练流式Transformer-Transducer（TT）模型，无需大规模数据与计算资源。相比传统的两阶段预训练+微调方法，该方案仅需单阶段训练即可获得鲁棒性能。我们对基于伪标签的流式TT模型进行了全面消融实验，包括：(1) n-gram语言模型浅融合的影响，(2) 命名实体上下文偏置，(3) 分块解码对低延迟应用的支持，(4) 模型性能随FSM规模的变化。结果表明，即使伪标签噪声较大，TT仍可成功从头训练。我们在CommonVoice的6种语言上验证了该框架，并提出多种启发式方法过滤幻觉伪标签。
原文摘要 · Abstract (English)
The training of automatic speech recognition (ASR) with little to no supervised data remains an open question. In this work, we demonstrate that streaming Transformer-Transducer (TT) models can be trained from scratch in consumer and accessible GPUs in their entirety with pseudo-labeled (PL) speech from foundational speech models (FSM). This allows training a robust ASR model just in one stage and does not require large data and computational budget compared to the two-step scenario with pre-training and fine-tuning. We perform a comprehensive ablation on different aspects of PL-based streaming TT models such as the impact of (1) shallow fusion of n-gram LMs, (2) contextual biasing with named entities, (3) chunk-wise decoding for low-latency streaming applications, and (4) TT overall performance as the function of the FSM size. Our results demonstrate that TT can be trained from scratch without supervised data, even with very noisy PLs. We validate the proposed framework on 6 languages from CommonVoice and propose multiple heuristics to filter out hallucinated PLs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。