让语音对话系统提前预测并使用工具,显著降低延迟并提升准确率。
Stream RAG: Instant and Accurate Spoken Dialogue Systems with Streaming Tool Usage
- 在用户说话时并行预测工具查询,提前准备答案。
- 问答准确率从11.1%提升至34.2%,工具延迟降低20%。
- 适用于语音与文本输入,适合实时交互类AI助手。
端到端语音输入语音输出对话系统正成为传统ASR-LLM-TTS流水线的有力替代,能生成更自然、更具表现力的响应,且延迟显著更低。然而,这些系统因缺乏事实依据仍易产生幻觉。尽管文本对话系统通过集成网络搜索、知识图谱等工具缓解此问题,我们首次将工具调用直接引入语音对话系统。关键挑战在于工具集成会大幅增加响应延迟,影响对话流畅性。为此,我们提出Streaming RAG框架,通过在用户说话过程中并行预测工具查询,提前生成结果,从而降低用户感知延迟。具体而言,我们设计了一套后训练流程,教会模型在语音进行中何时调用工具,并生成融合音频查询与检索结果的口语化摘要,兼顾准确性和响应速度。为评估该方法,我们构建了AudioCRAG基准数据集,将公开的CRAG数据集中的文本查询转换为语音形式。实验表明,该方法使问答准确率相对提升200%(从11.1%增至34.2%绝对值),同时将工具使用延迟降低20%。重要的是,Streaming RAG具有模态无关性,可同样应用于文本输入,推动更智能、实时的AI助理发展。
原文摘要 · Abstract (English)
End-to-end speech-in speech-out dialogue systems are emerging as a powerful alternative to traditional ASR-LLM-TTS pipelines, generating more natural, expressive responses with significantly lower latency. However, these systems remain prone to hallucinations due to limited factual grounding. While text-based dialogue systems address this challenge by integrating tools such as web search and knowledge graph APIs, we introduce the first approach to extend tool use directly into speech-in speech-out systems. A key challenge is that tool integration substantially increases response latency, disrupting conversational flow. To mitigate this, we propose Streaming Retrieval-Augmented Generation (Streaming RAG), a novel framework that reduces user-perceived latency by predicting tool queries in parallel with user speech, even before the user finishes speaking. Specifically, we develop a post-training pipeline that teaches the model when to issue tool calls during ongoing speech and how to generate spoken summaries that fuse audio queries with retrieved text results, thereby improving both accuracy and responsiveness. To evaluate our approach, we construct AudioCRAG, a benchmark created by converting queries from the publicly available CRAG dataset into speech form. Experimental results demonstrate that our streaming RAG approach increases QA accuracy by up to 200% relative (from 11.1% to 34.2% absolute) and further enhances user experience by reducing tool use latency by 20%. Importantly, our streaming RAG approach is modality-agnostic and can be applied equally to typed input, paving the way for more agentic, real-time AI assistants.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。