提出新框架，让关键词识别模型更快更稳训练，支持零样本新增关键词。
GE2E-KWS: Generalized End-to-End Training and Evaluation for Zero-shot Keyword Spotting
- 用嵌入中心对比法优化训练，提升收敛速度与稳定性。
- 419KB模型相对7.5GB大模型提升23.6%准确率，比同类模型高60.7% AUC。
- 模型可流式运行、内存小，无需重训即可支持新关键词识别。
我们提出GE2E-KWS——一种面向定制化关键词检测的通用端到端训练与评估框架。具体而言，将训练批次中的注册语音按关键词分组，计算其嵌入中心，并与所有测试语音嵌入对比以生成损失，模拟实际运行时的注册与验证流程。相比现有最优三元组损失方法，该方法通过优化矩阵运算提升了收敛稳定性和训练速度。为实现可靠模型评估，我们设计了模拟生产环境的评测流程，并直接衡量关键词匹配准确率。使用GE2E损失训练的419KB量化Conformer模型，在相对AUC上超越7.5GB ASR编码器23.6%，且比同规模三元组损失模型高出60.7%。所提关键词检测模型具备原生流式能力、低内存占用，可无需重训持续在设备端运行，支持零样本新增关键词。
原文摘要 · Abstract (English)
We propose GE2E-KWS -- a generalized end-to-end training and evaluation framework for customized keyword spotting. Specifically, enrollment utterances are separated and grouped by keywords from the training batch and their embedding centroids are compared to all other test utterance embeddings to compute the loss. This simulates runtime enrollment and verification stages, and improves convergence stability and training speed by optimizing matrix operations compared to SOTA triplet loss approaches. To benchmark different models reliably, we propose an evaluation process that mimics the production environment and compute metrics that directly measure keyword matching accuracy. Trained with GE2E loss, our 419KB quantized conformer model beats a 7.5GB ASR encoder by 23.6% relative AUC, and beats a same size triplet loss model by 60.7% AUC. Our KWS models are natively streamable with low memory footprints, and designed to continuously run on-device with no retraining needed for new keywords (zero-shot).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。