提出自适应聚类阈值，让设备端语音说话人分离更快更准。
Fast and Robust On-Device Speaker Diarization: Relative Minimum Cluster Size for Stride-Accelerated Pipelines
- 用可变最小聚类大小替代固定值，适配不同录音的嵌入数量。
- 在VoxConverse上将错误率从0.113降至0.079，恢复约89%精度。
- 在MPS上实现最高12.2倍加速，适合资源受限的设备部署。
语音应用如会议转录和语音助手需要设备端说话人分离，但推理成本限制了实际应用。本文研究基于Pyannote 3.1的流水线在消费级硬件（RTX 5070 Ti GPU和Apple M4笔记本）上的加速潜力，同时保持说话人分离错误率（DER）。简单策略：粗粒度分段步长与每块嵌入，可在AMI数据集上实现多倍提速且保持DER不变，但在真实场景数据VoxConverse上，DER从0.075升至0.113。我们发现失败原因是聚类阶段的说话人欠计数，源于固定最小聚类大小与每说话人嵌入数减少的交互作用。为此提出相对最小聚类大小：mcs = round(f * n)，其中f = 0.01，根据每录音嵌入预算动态调整。单一f值使VoxConverse DER恢复至0.079（约挽回89%精度损失），同时在AMI上保持稳定，加速流水线在MPS上相较CAM++基线最高达12.2倍提速。
原文摘要 · Abstract (English)
Speech applications such as meeting transcription and voice agents would benefit from on-device speaker diarization, but practical adoption is limited by inference cost. We study how far a Pyannote 3.1-based pipeline can be accelerated on consumer hardware (an RTX 5070 Ti GPU and an Apple M4 laptop) while preserving diarization error rate (DER). A simple recipe: coarser segmentation stride and per-chunk embedding, yields multi-fold speedups and is DER-neutral on AMI, but degrades sharply on in-the-wild data: on VoxConverse, DER rises from 0.075 to 0.113. We trace the failure to speaker under-counting in the clustering stage, caused by a fixed minimum cluster size interacting with the reduced number of embeddings per speaker. We propose a relative minimum cluster size, mcs = round(f * n) with f = 0.01, which adapts to the embedding budget per recording. A single value of f recovers VoxConverse DER to 0.079 (about 89% of the lost accuracy) while keeping AMI flat, and the accelerated pipeline reaches up to 12.2x speedup on AMI (MPS) over our CAM++ baseline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。