arXiv:2604.08412cs.SDcs.AI2026-04

通过历史交互建模语音路由,提升设备端语音检测精度

Selective Attention System (SAS): Device-Addressed Speech Detection for Real-Time On-Device Voice AI

  • 将语音检测视为基于交互历史的序列路由问题
  • 纯音频模型F1达0.86,融合视频可达0.95
  • 短时交互历史对决策至关重要,适合边缘设备部署

在预ASR边缘部署约束下,研究设备定向语音检测,系统需在转录前决定是否转发音频,且受严格延迟和计算限制。我们发现,在多人说话且语音时间模糊的场景中,该任务更宜建模为基于交互历史的序列路由问题,而非局部语句分类。为此提出序贯设备定向路由(SDAR)形式化框架,并实现可全在设备运行的精选注意力系统(SAS)。在60小时多说话人英文测试集上,纯音频配置达F1=0.86(精确率0.89,召回率0.83);加入摄像头后,音视频融合使F1提升至0.95(精确率0.97,召回率0.93)。移除因果交互历史(阶段3)导致音视频配置下F1降至0.57±0.03,为所有测试组件中影响最大,表明短时交互历史在该设定下蕴含重要决策信息。SAS可在ARM Cortex-A级硬件上全本地运行,延迟低于150毫秒,内存占用小于20 MB。所有结果基于内部评估的专有数据集,主要在英语环境下验证;第8.8节提及可共享5小时评估子集供独立验证。

原文摘要 · Abstract (English)

We study device-addressed speech detection under pre-ASR edge deployment constraints, where systems must decide whether to forward audio before transcription under strict latency and compute limits. We show that, in multi-speaker environments with temporally ambiguous utterances, this task is more effectively modelled as a sequential routing problem over interaction history than as an utterance-local classification task. We formalize this as Sequential Device-Addressed Routing (SDAR) and present the Selective Attention System (SAS), an on-device implementation that instantiates this formulation. On a held-out 60-hour multi-speaker English test set, the primary audio-only configuration achieves F1=0.86 (precision=0.89, recall=0.83); with an optional camera, audio+video fusion raises F1 to 0.95 (precision=0.97, recall=0.93). Removing causal interaction history (Stage~3) reduced F1 from 0.95 to 0.57+/-0.03 in the audio+video configuration under our evaluation protocol. Among the tested components, this was the largest observed ablation effect, indicating that short-horizon interaction history carries substantial decision-relevant information in the evaluated setting. SAS runs fully on-device on ARM Cortex-A class hardware (<150 ms latency, <20 MB footprint). All results are from internal evaluation on a proprietary dataset evaluated primarily in English; a 5-hour evaluation subset may be shared for independent verification (Section 8.8).

语音检测边缘计算多说话人序列路由

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。