解决跨模型语言识别合并难题，提升隐私保护语音识别效果。
Federated Heterogeneous Language Model Optimization for Hybrid Automatic Speech Recognition
- 提出匹配-合并框架，融合神经与非神经语言模型。
- 新算法使错误率降低，收敛速度比旧方法快7倍。
- 适合需要隐私保护的多设备语音系统部署。
语音识别模型训练越来越依赖去中心化的联邦学习以保障数据隐私，产生多个本地模型需有效融合。在混合语音识别系统中，声学模型可借助成熟方法合并，但语言模型（用于重评分候选词列表）因非神经n-gram模型与神经网络模型的异构性面临挑战。本文提出异构语言模型优化任务，引入匹配-合并范式及两种算法：基于遗传操作的遗传匹配-合并算法（GMMA），以及利用强化学习实现高效收敛的强化匹配-合并算法（RMMA）。在七个OpenSLR数据集上的实验表明，RMMA平均字符错误率最低，泛化性能优于基线，收敛速度最高达GMMA的七倍，凸显该范式在可扩展、隐私保护语音识别系统中的潜力。
原文摘要 · Abstract (English)
Training automatic speech recognition (ASR) models increasingly relies on decentralized federated learning to ensure data privacy and accessibility, producing multiple local models that require effective merging. In hybrid ASR systems, while acoustic models can be merged using established methods, the language model (LM) for rescoring the N-best speech recognition list faces challenges due to the heterogeneity of non-neural n-gram models and neural network models. This paper proposes a heterogeneous LM optimization task and introduces a match-and-merge paradigm with two algorithms: the Genetic Match-and-Merge Algorithm (GMMA), using genetic operations to evolve and pair LMs, and the Reinforced Match-and-Merge Algorithm (RMMA), leveraging reinforcement learning for efficient convergence. Experiments on seven OpenSLR datasets show RMMA achieves the lowest average Character Error Rate and better generalization than baselines, converging up to seven times faster than GMMA, highlighting the paradigm's potential for scalable, privacy-preserving ASR systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。