解决语音关键词识别在边缘设备上的遗忘问题,无需额外存储。
Dark Experience for Incremental Keyword Spotting
- 用记忆缓存中的真实标签和模型输出,持续蒸馏旧知识
- 在谷歌语音命令数据集上平均准确率超越现有方法
- 适合资源受限的嵌入式语音设备,无需任务标识
语音关键词检测(KWS)对识别音频中的关键词至关重要,广泛应用于Apple Siri和Google Home等边缘设备。当前基于深度学习的KWS系统通常在有限关键词上训练,面对新领域时性能下降,常通过少样本微调适应,但易引发灾难性遗忘。尽管渐进式持续学习(CL)策略被提出以缓解此问题,仍存在需任务ID信息、存储开销大等局限,不适用于轻量级设备。为此,我们提出暗知识关键词检测(DE-KWS),一种新颖的CL方法,利用暗知识在整个训练过程中蒸馏过往经验。DE-KWS结合回放与蒸馏,使用内存缓冲区中存储的真实标签和软标签,保持跨任务性能。在谷歌语音命令数据集上的评估显示,DE-KWS在不增加模型大小的前提下,平均准确率优于现有基线,为资源受限的边缘设备提供了有效解决方案。代码已开源至GitHub,供后续研究使用。
原文摘要 · Abstract (English)
Spoken keyword spotting (KWS) is crucial for identifying keywords within audio inputs and is widely used in applications like Apple Siri and Google Home, particularly on edge devices. Current deep learning-based KWS systems, which are typically trained on a limited set of keywords, can suffer from performance degradation when encountering new domains, a challenge often addressed through few-shot fine-tuning. However, this adaptation frequently leads to catastrophic forgetting, where the model's performance on original data deteriorates. Progressive continual learning (CL) strategies have been proposed to overcome this, but they face limitations such as the need for task-ID information and increased storage, making them less practical for lightweight devices. To address these challenges, we introduce Dark Experience for Keyword Spotting (DE-KWS), a novel CL approach that leverages dark knowledge to distill past experiences throughout the training process. DE-KWS combines rehearsal and distillation, using both ground truth labels and logits stored in a memory buffer to maintain model performance across tasks. Evaluations on the Google Speech Command dataset show that DE-KWS outperforms existing CL baselines in average accuracy without increasing model size, offering an effective solution for resource-constrained edge devices. The scripts are available on GitHub for the future research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。