用伪标签提升流式语音识别在新领域的表现
StreamHear: Domain-Adapted Pseudo-Labeling for Semi-Supervised Streaming Speech Recognition
- 先用标注数据微调离线教师模型，生成未标注数据的伪标签
- 在四个数据集上显著优于监督微调，接近离线教师性能
- 适合标注成本高、未标注数据多的语音识别场景
流式自动语音识别（ASR）在领域迁移的音频上表现不佳，其中领域内标注数据成本高，而未标注音频却大量存在。我们提出 StreamHear，一种半监督流程：先在标注训练集上微调一个离线转换器教师模型，生成未标注部分的伪标签，再将伪标签与真实标签混合，用于微调流式学生模型。我们进一步引入基于先验的动态规划重对齐步骤，利用ASR假设作为锚点，修正分块级词位错位问题。在涵盖金融通话、朗读语料和电话质量对话的四个数据集上，StreamHear 均持续优于监督学生微调，缩小了与离线教师模型的差距。
原文摘要 · Abstract (English)
Streaming automatic speech recognition (ASR) underperforms on domain-shifted target audio, where labeled in-domain data is costly to prepare while unlabeled audio is abundant. We present StreamHear, a semi-supervised pipeline that adapts a pretrained streaming student by fine-tuning an offline transducer teacher on the labeled training set, generating pseudo-labels on the unlabeled portion, and fine-tuning the student on the mixture. We further introduce a prior-regularized dynamic-programming realignment step that fixes chunk-level word placement using an ASR-hypothesis anchor. Across four datasets spanning financial calls, prepared read speech, and phone-quality dialogue, StreamHear consistently outperforms supervised student fine-tuning and narrows the gap to the offline teacher.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。