用双路径推测生成，实现实时对话低延迟高质输出
RelayS2S: A Dual-Path Speculative Generation for Real-Time Dialogue
- 并行运行快速S2S草稿与慢速ASR+LLM生成，实时响应
- P90延迟接近S2S模型，平均质量保留99%的级联模型水平
- 无需修改现有系统，可轻量集成到任意级联架构中
实时语音对话系统面临延迟与响应质量的矛盾。端到端语音到语音（S2S）模型响应快，自然处理换言、附和与打断，但语义较弱；级联流水线（ASR → LLM）质量高，但延迟随模型增大而上升。我们提出RelayS2S，一种双路径推测生成架构：在检测到发言后并行运行两条路径。快速路径——双工S2S模型——即时生成短响应前缀并流式传输至语音合成，同时持续监听实时音频事件；慢速路径——级联ASR→LLM管道——基于已确定前缀生成高质量后续内容，实现无缝语音输出。一个轻量级学习验证器控制前缀交接，适时提交或优雅回退至慢速路径。实验表明，RelayS2S在保持99%级联模型平均得分的同时，达到与S2S模型相当的P90响应延迟，且随着慢速路径模型规模增大，优势更明显。由于前缀交接无需修改任一组件，RelayS2S可作为轻量级、即插即用的模块，集成至现有级联系统。代码与数据公开于：https://github.com/mailong25/relays2s
原文摘要 · Abstract (English)
Real-time spoken dialogue systems face a fundamental tension between latency and response quality. End-to-end speech-to-speech (S2S) models respond immediately and naturally handle turn-taking, backchanneling, and interruption, but produce semantically weaker outputs. Cascaded pipelines (ASR -> LLM) deliver stronger responses at the cost of latency that grows with model size. We present RelayS2S, a hybrid architecture that runs two paths in parallel upon turn detection. The fast path -- a duplex S2S model -- speculatively drafts a short response prefix that is streamed immediately to TTS for low-latency audio onset, while continuing to monitor live audio events. The slow path -- a cascaded ASR -> LLM pipeline -- generates a higher-quality continuation conditioned on the committed prefix, producing a seamless utterance. A lightweight learned verifier gates the handoff, committing the prefix when appropriate or falling back gracefully to the slow path alone. Experiments show that RelayS2S achieves P90 onset latency comparable to the S2S model while retaining 99% cascaded response quality in average score, with benefits growing as the slow-path model scales. Because the prefix handoff requires no architectural modification to either component, RelayS2S serves as a lightweight, drop-in addition to existing cascaded pipelines. Our code and data are publicly available at: https://github.com/mailong25/relays2s
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。