用语义推理替代静音判断,让语音助手更准识别用户是否说完。
The Trade-off Was in the Labels: Causal Supervision for Turn-Aware Streaming ASR

- 基于输入内容实时计算决策标签,避免未来信息泄露
- 0.39秒延迟下实现0.97边界召回率,误报仅0.3次/分钟
- 适合需要精准断句的语音交互场景,如客服机器人
语音助手需实时判断用户是否结束发言,但静音常不可靠:读号码时停顿、单字“停止”结束对话、长问题内部停顿甚至超过真实话语间隙。现有方案依赖语音活动检测加静音超时,无法区分这些情况,因句内停顿常超过句间间隙;真正区别在于已说出内容是否构成完整语义——这正是识别器生成转录时所计算的。本文首次提出面向流式语音识别的端到端训练方案与基准测试:在Qwen3-ASR-0.6B上仅用一个小型LoRA适配器,在单张GPU上训练数小时,即可同时完成转录、基于语义和静音的断句、口述文本处理,并结合上下文进行理解。在匹配部署条件的测试中,实现0.97的边界召回率,中位延迟0.39秒,每分钟仅0.3次误触发,且在新测试集上可复现;无静音超时系统无法达到此性能。方法核心原则:每个流式决策标签必须仅由决策前输入可计算得出。离线数据集违背此原则,包含未来信息,导致模型产生振荡及虚假的召回-精度权衡;当额外添加1秒静音后,‘损坏’模型的断句召回从0.10跃升至1.00。类似信息泄漏也出现在上下文处理中:始终匹配的引导前缀导致40%的干扰;而使用与音频矛盾的反事实样本将其降至0.8%,同时保留+28.9个百分点的实体识别提升收益。
原文摘要 · Abstract (English)
A voice agent must decide, moment to moment, whether the user has finished; silence rarely settles it: a caller reading a phone number pauses mid-digits, a one-word "Stop!" ends a turn, a long question carries pauses longer than real turn-gaps. A voice-activity detector plus a silence timeout (the deployed default) cannot separate these, because within-turn pauses routinely exceed between-turn gaps; what distinguishes them is whether the words so far form a complete thought: what a recognizer computes to produce a transcript. We present the first open training recipe and benchmark for turn-aware streaming ASR: a small LoRA adapter on Qwen3-ASR-0.6B, trained in hours on one GPU, that transcribes, detects end-of-turn from meaning and silence, handles dictation, and grounds transcription in context. On a deployment-matched benchmark it reaches 0.97 boundary recall at 0.39 s median latency with 0.3 false fires per speech-minute, replicated on a fresh test set; no silence timeout reaches this point. The recipe rests on one principle: every streaming-decision label must be computable from input up to the decision point. Offline corpora violate it, encoding the future; such clairvoyant labels manufactured oscillation and a phantom recall-versus-precision trade-off, exposed when one appended second of silence raised a "broken" model's end-of-turn recall from 0.10 to 1.00. The same leak recurred with context: an always-matching biasing prefix became a copied shortcut (40% intrusion), and counterfactuals disagreeing with the audio cut this to 0.8% while keeping most of a +28.9 pp entity-recall benefit.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。