用文本嵌入增强语音模型，少样本下精准识别自定义关键词
Text-Aware Adapter for Few-Shot Keyword Spotting
- 通过文本编码器生成关键词嵌入，指导语音模型微调
- 仅增加0.14%参数，35个关键词识别率显著提升
- 适合资源有限场景下的个性化语音唤醒应用
近期灵活关键词识别（KWS）技术允许用户通过输入文本而非语音进行关键词注册。然而，针对特定关键词的识别性能仍有提升空间。本文提出一种新型少样本迁移学习方法——文本感知适配器（TA-adapter），用于在仅有少量语音样本的情况下，增强预训练的灵活KWS模型对特定关键词的识别能力。为适应声学编码器，该方法利用联合预训练的文本编码器生成关键词的文本嵌入，作为关键词的代表性向量。仅微调网络中极小部分参数，同时保持核心权重不变，使模型高效且可无缝恢复至原始预训练状态。实验表明，在Google Speech Commands V2数据集上的35个不同关键词上，该方法均取得显著性能提升，总参数量仅增加0.14%。
原文摘要 · Abstract (English)
Recent advances in flexible keyword spotting (KWS) with text enrollment allow users to personalize keywords without uttering them during enrollment. However, there is still room for improvement in target keyword performance. In this work, we propose a novel few-shot transfer learning method, called text-aware adapter (TA-adapter), designed to enhance a pre-trained flexible KWS model for specific keywords with limited speech samples. To adapt the acoustic encoder, we leverage a jointly pre-trained text encoder to generate a text embedding that acts as a representative vector for the keyword. By fine-tuning only a small portion of the network while keeping the core components' weights intact, the TA-adapter proves highly efficient for few-shot KWS, enabling a seamless return to the original pre-trained model. In our experiments, the TA-adapter demonstrated significant performance improvements across 35 distinct keywords from the Google Speech Commands V2 dataset, with only a 0.14% increase in the total number of parameters.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。