arXiv:2608.17972cs.SD2026-08

实现10毫秒级低延迟目标说话人识别,适合助听器实时增强。

Target Speaker Identification: A Low-Latency Streaming Pipeline

论文配图:Target Speaker Identification: A Low-Latency Streaming Pipeline
图 1 · 摘自论文原文
  • 分两步:先流式语音分离,再与目标说话人比对验证。
  • 17次测试中准确率超90%,特异性达95%-98%。
  • 专为助听器设计,适合需要快速响应的场景。

我们提出一个基于开源预训练模型的实时流式目标说话人识别系统,针对助听器应用中10毫秒级延迟即可感知的需求。采用两阶段方法:首先使用低延迟流式说话人分离技术对输入音频进行分割,然后与注册的目标说话人进行说话人验证。为模拟对话语音并减少重叠,选用《这是美国生活》播客语料库,并将主持人作为一致的目标说话人。在离线说话人分离方面,以说话人分离错误率(DER)为指标对比Pyannote和LIUM,选择性能更优且支持流式的Pyannote。随后使用Pyannote和TitaNet-Large评估说话人验证,生成ROC曲线以确定工作区间。集成Diart并调整聚类参数,在保持实时运行的同时降低DER。最终将Diart与Pyannote验证结合,通过将预测和真实语音区域转换为100毫秒二值掩码评估系统性能。在17个测试片段中,系统在余弦距离阈值0.7-0.75下,中位准确率超过0.90,特异性达0.95-0.98,证明了下游低延迟定向放大在实际应用中的可行性。

原文摘要 · Abstract (English)

We present a real-time pipeline of open source, pretrained models for streaming identification of a target speaker, motivated by hearing-aid applications where latency as low as 10 ms can be perceptible. We formulate a two-step approach in which incoming audio is first segmented by speaker using low-latency streaming diarization, followed by speaker verification against a registered target speaker. To emulate conversational speech while minimizing overlap, we use the This American Life Podcast Transcripts dataset and select the host as a consistent target speaker. We benchmark offline diarization with Pyannote and LIUM using diarization error rate (DER) and select Pyannote based on baseline performance and compatibility with streaming. We then evaluate speaker verification using Pyannote and TitaNet-Large and generate ROC curves to select an operating region. We integrate Diart and tune clustering parameters to reduce DER while maintaining real-time operation. We pair Diart with Pyannote verification and evaluate system-level performance by converting predicted and ground-truth speech regions into 100 ms binary masks. Across 17 evaluation episodes, the system achieves greater than 0.90 median accuracy with high specificity (0.95-0.98) at cosine distance thresholds of 0.7-0.75, demonstrating a practical proof of concept for downstream low-latency selective amplification.

说话人识别低延迟助听器流式处理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。