用大模型结合上下文与语音识别置信度，提升智能助手后续对话的精准唤醒。
Device-Directed Speech Detection for Follow-up Conversations Using Large Language Models
- 用大模型理解前一句对话，推断后续对话是否应由设备响应
- 在真实数据集上降低20%-40%误触发率，固定漏检率10%时效果显著
- 适合做智能助手、语音交互系统优化的研究者和工程师参考
与虚拟助手的后续对话可让用户无需重复唤醒即可自然续聊。因此，准确检测后续对话是否应由设备响应（即设备定向语音检测，DDSD）至关重要。本文探索使用大语言模型（LLM），在判断后续对话时，基于先前查询的语音识别解码文本进行推理，通过提示预训练LLM或在其基础上微调二分类器实现。同时，我们利用语音识别的不确定性来设计提示词。在真实世界后续对话数据集上的实验表明，相比仅单独建模后续对话，该方法通过联合建模历史语音上下文与语音识别置信度，使误触发率降低20%-40%（在固定漏检率为10%的条件下）。
原文摘要 · Abstract (English)
Follow-up conversations with virtual assistants (VAs) enable a user to seamlessly interact with a VA without the need to repeatedly invoke it using a keyword (after the first query). Therefore, accurate Device-directed Speech Detection (DDSD) from the follow-up queries is critical for enabling naturalistic user experience. To this end, we explore the notion of Large Language Models (LLMs) and model the first query when making inference about the follow-ups (based on the ASR-decoded text), via prompting of a pretrained LLM, or by adapting a binary classifier on top of the LLM. In doing so, we also exploit the ASR uncertainty when designing the LLM prompts. We show on the real-world dataset of follow-up conversations that this approach yields large gains (20-40% reduction in false alarms at 10% fixed false rejects) due to the joint modeling of the previous speech context and ASR uncertainty, compared to when follow-ups are modeled alone.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。