基于到达时间的缓存机制，实现低延迟实时说话人分离。
Streaming Sortformer: Speaker Cache-Based Online Speaker Diarization with Arrival-Time Ordering
- 用到达时间顺序缓存说话人声学特征，动态更新高分帧。
- 每说话人缓存数量自适应调整，提升跟踪精度与缓存效率。
- 适合实时多说话人场景，尤其低延迟语音处理应用。
本文提出一种流式扩展的Sortformer说话人分离框架，其核心特性为输出说话人的到达时间排序。所提方法采用到达时间顺序说话人缓存（AOSC），存储先前观测到的说话人帧级声学嵌入。与传统说话人追踪缓冲区不同，AOSC按说话人到达时间索引顺序排列嵌入，并通过模型历史预测得分最高的帧动态更新。值得注意的是，每个说话人存储的嵌入数量由更新机制动态决定，确保缓存高效利用与精确跟踪。在基准数据集上的实验验证了该方法的有效性与灵活性，即使在低延迟设置下表现优异。结果表明，Streaming Sortformer是实时多说话人跟踪的稳健解决方案，为流式多人语音处理奠定了基础。
原文摘要 · Abstract (English)
This paper presents a streaming extension for the Sortformer speaker diarization framework, whose key property is the arrival-time ordering of output speakers. The proposed approach employs an Arrival-Order Speaker Cache (AOSC) to store frame-level acoustic embeddings of previously observed speakers. Unlike conventional speaker-tracing buffers, AOSC orders embeddings by speaker index corresponding to their arrival time order, and is dynamically updated by selecting frames with the highest scores based on the model's past predictions. Notably, the number of stored embeddings per speaker is determined dynamically by the update mechanism, ensuring efficient cache utilization and precise speaker tracking. Experiments on benchmark datasets confirm the effectiveness and flexibility of our approach, even in low-latency setups. These results establish Streaming Sortformer as a robust solution for real-time multi-speaker tracking and a foundation for streaming multi-talker speech processing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。