无需录音即可识别对话主讲人，实时准确率提升明显。
EEND-SAA: Enrollment-Less Main Speaker Voice Activity Detection Using Self-Attention Attractors
- 用自注意力吸引子捕捉说话连续性和音量特征，自动定位主讲人。
- 在多说话人语音混合场景中，主讲人错误率降至3.61%，F1达0.9818。
- 适合会议、客服等未知主讲人的真实语音场景，支持流式处理。
语音活动检测（VAD）是语音系统的关键，但传统方法仅判断是否有语音而无法识别说话人。目标说话人VAD（TS-VAD）虽可通过短段录音识别特定说话人，但在会议或客服等开放场景中因主讲人未知而失效。本文提出EEND-SAA，一种无需录音、支持流式处理的主讲人语音活动检测框架，通过分析语音连续性与音量，自动识别更稳定清晰的主讲人。模型基于EEND，采用两个自注意力吸引子构建Transformer结构，并引入因果掩码实现实时推理。在多说话人LibriSpeech混合数据集上的实验表明，相较于SA-EEND基线，EEND-SAA将主讲人错误率从6.63%降低至3.61%，F1值从0.9667提升至0.9818，在存在说话人重叠和噪声条件下达到当前最优性能。
原文摘要 · Abstract (English)
Voice activity detection (VAD) is essential in speech-based systems, but traditional methods detect only speech presence without identifying speakers. Target-speaker VAD (TS-VAD) extends this by detecting the speech of a known speaker using a short enrollment utterance, but this assumption fails in open-domain scenarios such as meetings or customer service calls, where the main speaker is unknown. We propose EEND-SAA, an enrollment-less, streaming-compatible framework for main-speaker VAD, which identifies the primary speaker without prior knowledge. Unlike TS-VAD, our method determines the main speaker as the one who talks more steadily and clearly, based on speech continuity and volume. We build our model on EEND using two self-attention attractors in a Transformer and apply causal masking for real-time use. Experiments on multi-speaker LibriSpeech mixtures show that EEND-SAA reduces main-speaker DER from 6.63% to 3.61% and improves F1 from 0.9667 to 0.9818 over the SA-EEND baseline, achieving state-of-the-art performance under conditions involving speaker overlap and noise.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。