提出协作推理框架，让语音助手更准判断用户是否说完。
Speculative End-Turn Detector for Efficient Speech Chatbot Assistant
- 轻量GRU模型本地实时检测停顿，服务器用Wav2vec做精准分类。
- 在资源受限设备上实现高精度端到端对话轮次检测，延迟更低。
- 首次公开端轮次检测数据集，适合语音交互系统开发者参考。
基于大语言模型的语音对话系统在理解人类语音和生成恰当回应方面表现出色，但在端轮次检测（ETD）——区分用户发言结束与犹豫停顿——方面存在挑战，常导致响应过早或延迟，破坏对话流畅性。本文提出首个公开的端轮次检测数据集ETD Dataset，包含由文本转语音模型生成的合成语音和从网络收集的真实语音数据。同时提出SpeculativeETD，一种新型协作推理框架，在资源受限环境中平衡效率与准确性。该方法结合本地设备上的轻量级GRU模型实时检测非发声单元，以及服务器端高性能的Wav2vec模型进行复杂分类，以区分真正结束与短暂停顿。实验表明，SpeculativeETD显著提升ETD准确率，同时保持低计算开销。数据集与代码将在审稿后公开。
原文摘要 · Abstract (English)
Spoken dialogue systems powered by large language models have demonstrated remarkable abilities in understanding human speech and generating appropriate spoken responses. However, these systems struggle with end-turn detection (ETD) -- the ability to distinguish between user turn completion and hesitation. This limitation often leads to premature or delayed responses, disrupting the flow of spoken conversations. In this paper, we introduce the ETD Dataset, the first public dataset for end-turn detection. The ETD dataset consists of both synthetic speech data generated with text-to-speech models and real-world speech data collected from web sources. We also propose SpeculativeETD, a novel collaborative inference framework that balances efficiency and accuracy to improve real-time ETD in resource-constrained environments. Our approach jointly employs a lightweight GRU-based model, which rapidly detects the non-speaking units in real-time on local devices, and a high-performance Wav2vec-based model running on the server to make a more challenging classification of distinguishing turn ends from mere pauses. Experiments demonstrate that the proposed SpeculativeETD significantly improves ETD accuracy while keeping the required computations low. Datasets and code will be available after the review.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。