用小模型从预训练语音模型中提取抗噪离散单元，提升嘈杂环境下的语音识别效果。
Efficient Extraction of Noise-Robust Discrete Units from Self-Supervised Speech Models
- 用轻量编码器-解码器同时去噪并离散化语音特征
- 在噪声环境下离散化与语音识别任务上优于多种预训练方法
- 仅需少量目标环境无标签数据即可适配新场景
连续语音可通过自监督学习（SSL）语音模型的隐藏特征提取离散单元。尽管SSL模型规模越来越大、训练数据越来越多，但对真实场景中的加性噪声或混响等畸变仍敏感，导致离散单元偏移。本文提出一种参数高效的方法：通过训练小型编码器-解码器模型（可选适配器），对预训练SSL模型的隐藏特征进行去噪和离散化。该模型能根据SSL特征为带噪语音生成干净的离散序列。所提去噪器在噪声条件下的离散化与语音识别任务中表现优于多种预训练方法，并可在仅有少量目标环境无标签录音的情况下微调以适应新场景。
原文摘要 · Abstract (English)
Continuous speech can be converted into a discrete sequence by deriving discrete units from the hidden features of self-supervised learned (SSL) speech models. Although SSL models are becoming larger and trained on more data, they are often sensitive to real-life distortions like additive noise or reverberation, which translates to a shift in discrete units. We propose a parameter-efficient approach to generate noise-robust discrete units from pre-trained SSL models by training a small encoder-decoder model, with or without adapters, to simultaneously denoise and discretise the hidden features of the SSL model. The model learns to generate a clean discrete sequence for a noisy utterance, conditioned on the SSL features. The proposed denoiser outperforms several pre-training methods on the tasks of noisy discretisation and noisy speech recognition, and can be finetuned to the target environment with a few recordings of unlabeled target data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。