用关键词生成专属滤波器,小模型也能精准听懂陌生词汇。
Keyword Spotting with Hyper-Matched Filters for Small Footprint Devices
- 用关键词生成唯一卷积权重,形成专用匹配滤波器。
- 420万参数模型在陌生语言上仍保持高精度,优于大模型。
- 适合资源受限设备,支持新词快速部署,如语音助手。
开放词汇关键词检测(KWS)旨在识别语音中任意词汇,无论是否在训练数据中出现。本文提出一种面向小型设备的先进开放词汇关键词检测模型。模型由语音编码器、目标关键词编码器和检测网络构成。语音编码器采用tiny Whisper或tiny Conformer。关键词编码器为超网络,输入关键词字符序列后生成特定卷积层的权重,可视为关键词专属匹配滤波器。检测网络利用该滤波器进行关键词特异性卷积,并引导Perceiver模块的交叉注意力机制判断目标词是否存在。实验表明,本系统在小设备上达到当前最优检测性能,且对域外条件(包括第二语言语音)具有强泛化能力。尤其值得注意的是,最小模型仅420万参数,性能媲美甚至超越数倍更大的模型,兼具高效与鲁棒性。
原文摘要 · Abstract (English)
Open-vocabulary keyword spotting (KWS) refers to the task of detecting words or terms within speech recordings, regardless of whether they were included in the training data. This paper introduces an open-vocabulary keyword spotting model with state-of-the-art detection accuracy for small-footprint devices. The model is composed of a speech encoder, a target keyword encoder, and a detection network. The speech encoder is either a tiny Whisper or a tiny Conformer. The target keyword encoder is implemented as a hyper-network that takes the desired keyword as a character string and generates a unique set of weights for a convolutional layer, which can be considered as a keyword-specific matched filter. The detection network uses the matched-filter weights to perform a keyword-specific convolution, which guides the cross-attention mechanism of a Perceiver module in determining whether the target term appears in the recording. The results indicate that our system achieves state-of-the-art detection performance and generalizes effectively to out-of-domain conditions, including second-language (L2) speech. Notably, our smallest model, with just 4.2 million parameters, matches or outperforms models that are several times larger, demonstrating both efficiency and robustness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。