用微调前后嵌入差异提升儿童语音识别准确率
Mind the Shift: Using Delta SSL Embeddings to Enhance Child ASR
- 通过对比微调前后嵌入差异，提取任务特异性信息
- 融合差值嵌入使HuBERT相对词错误率降10%，W2V2降4.4%
- 新方法在MyST数据集上达到9.64%最低词错误率
自监督学习（SSL）模型在多个语音任务中表现优异，但儿童语音识别因数据稀缺和预训练领域不匹配仍具挑战。对儿童语音微调SSL模型会引发表征空间的偏移。本文假设：微调前后嵌入的差值（即delta SSL embeddings）编码了任务特异性信息，可补充另一SSL模型的微调特征。我们在MyST儿童语料库上评估多种融合策略。结果表明，使用WavLM的delta嵌入融合相比仅微调嵌入融合，使HuBERT相对词错误率降低10%，W2V2降低4.4%。尤为显著的是，将WavLM与delta W2V2嵌入融合后，词错误率达到9.64%，刷新了当前基于SSL模型在MyST数据集上的最优纪录。这些发现验证了差值嵌入的有效性，并凸显特征融合是推动儿童语音识别发展的有前景方向。
原文摘要 · Abstract (English)
Self-supervised learning (SSL) models have achieved impressive results across many speech tasks, yet child automatic speech recognition (ASR) remains challenging due to limited data and pretraining domain mismatch. Fine-tuning SSL models on child speech induces shifts in the representation space. We hypothesize that delta SSL embeddings, defined as the differences between embeddings from a finetuned model and those from its pretrained counterpart, encode task-specific information that complements finetuned features from another SSL model. We evaluate multiple fusion strategies on the MyST childrens corpus using different models. Results show that delta embedding fusion with WavLM yields up to a 10 percent relative WER reduction for HuBERT and a 4.4 percent reduction for W2V2, compared to finetuned embedding fusion. Notably, fusing WavLM with delta W2V2 embeddings achieves a WER of 9.64, setting a new state of the art among SSL models on the MyST corpus. These findings demonstrate the effectiveness of delta embeddings and highlight feature fusion as a promising direction for advancing child ASR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。