用纯文本数据提升语音识别模型效率，简化训练流程
Text-Utilization for Encoder-dominated Speech Recognition Models
- 通过模态对齐与动态下采样，将文本信息融入编码器
- 大编码器小解码器配置在LibriSpeech上性能更优
- 随机时长模型等简单方法比复杂方案更有效
本文研究如何高效利用纯文本数据提升语音识别性能，聚焦于以编码器为主的模型架构，该架构可实现更快的识别速度。我们系统比较了多种融合文本数据的技术，包括模态对齐和动态下采样，以在编码器中生成文本级表示。在LibriSpeech数据集上的实验表明，采用更大编码器搭配更小解码器的配置，性能可达到甚至超过传统大解码器结构。研究还发现，如随机时长模型等简单方案往往比复杂方法更具优势，显著简化了训练流程。所有代码与训练方案均已公开。
原文摘要 · Abstract (English)
This paper investigates efficient methods for utilizing text-only data to improve speech recognition, focusing on encoder-dominated models that facilitate faster recognition. We provide a comprehensive comparison of techniques to integrate text-only data, including modality matching and dynamic downsampling to reach text-level representations within the encoder. Our experiments on the LibriSpeech corpus show that a larger encoder with a smaller decoder can equal or surpass the performance of architectures with larger decoders. We demonstrate that simple configurations, such as random duration models, are often more effective than complex alternatives, significantly simplifying the training pipeline. All code and recipes are made publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。