用自适应知识蒸馏提升语音助手指令识别准确率
Adaptive Knowledge Distillation for Device-Directed Speech Detection
- 在冻结的预训练教师模型上加任务适配器，联合训练学生模型
- 关键词和非关键词唤醒场景下错误率分别降低26%和19%
- 适用于Transformer和Conformer架构，适合边缘设备部署
设备定向语音检测（DDSD）是一项二分类任务，用于区分用户对语音助手的指令与背景语音或旁白对话，对实现自然交互体验至关重要。为此，我们提出一种知识蒸馏（KD）方法，在保证高效部署的前提下提升DDSD准确率。具体而言，引入一种新颖的自适应知识蒸馏方法，从通用声学编码器（教师模型）中转移知识。在冻结的教师编码器基础上，添加任务特定适配器，与学生模型共同训练于DDSD任务。实验表明，该方法在关键词唤醒和无关键词（后续）唤醒场景下，分别相较无蒸馏的学生模型提升26%和19%的等错误率（Equal Error Rate）。此外，该方法在Transformer和Conformer两种模型架构上均表现良好，具备良好泛化性。
原文摘要 · Abstract (English)
Device-directed speech detection (DDSD) is a binary classification task that separates the user's queries to a voice assistant (VA) from background speech or side conversations. This is important for achieving naturalistic user experience. To this end, we propose knowledge distillation (KD) to enhance DDSD accuracy while ensuring efficient deployment. Specifically, we introduce a novel adaptive KD method that transfers knowledge from general representations of an ASR large pre-trained acoustic encoder (teacher). We apply task-specific adapters, on top of the (frozen) teacher encoder, trained jointly with the student model on DDSD. We demonstrate that the proposed adaptive KD outperforms the student model without distillation in the keyword and keyword-free (follow-up) invocations, with an improvement of +26% and +19% in terms of Equal Error Rate, respectively. We also show that this approach generalizes across the transformer and conformer-based model architectures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。