通过估计语音帧的不确定性,提升说话人识别的准确性和可靠性。
U3-xi: Pushing the Boundaries of Speaker Recognition by Incorporating Uncertainty
- 根据帧的不确定性动态加权,降低噪声干扰帧的影响。
- 在VoxCeleb1上相对降低21.1%的误报率和15.57%的最小代价。
- 适用于多种编码器,提升模型可解释性与鲁棒性。
话语级说话人嵌入通常通过聚合帧级表示获得。然而,在真实场景中,单个帧不仅包含说话人相关信息,还混杂多种干扰因素,导致不同帧对最终嵌入的贡献不均。为此,我们提出估计每帧的固有不确定性,并据此分配自适应权重,不确定性高的帧获得较低关注。基于此,我们提出U3-xi框架,旨在生成更可靠且可解释的说话人嵌入不确定性估计。具体包括:1)通过随机方差损失实现说话人级不确定性监督,以话语嵌入与对应说话人中心的距离作为伪标签;2)通过在训练中将预测不确定性注入softmax尺度,实现全局级不确定性监督,自适应调整决策边界锐度;3)重构不确定性估计模块,引入带有多视角自注意力的Transformer编码器,捕捉丰富的局部与长时依赖。大量实验表明,U3-xi具有模型无关性,可无缝集成于多种说话人编码器。应用于ECAPA-TDNN时,在VoxCeleb1测试集上相对提升21.1%(EER)和15.57%(minDCF)。
原文摘要 · Abstract (English)
An utterance-level speaker embedding is typically obtained by aggregating a sequence of frame-level representations. However, in real-world scenarios, individual frames encode not only speaker-relevant information but also various nuisance factors. As a result, different frames contribute unequally to the final utterance-level speaker representation for Automatic Speaker Verification systems. To address this issue, we propose to estimate the inherent uncertainty of each frame and assign adaptive weights accordingly, where frames with higher uncertainty receive lower attention. Based on this idea, we present U3-xi, a comprehensive framework designed to produce more reliable and interpretable uncertainty estimates for speaker embeddings. Specifically, we introduce several strategies for uncertainty supervision. First, we propose speaker-level uncertainty supervision via a Stochastic Variance Loss, where the distance between an utterance embedding and its corresponding speaker centroid serves as a pseudo ground truth for uncertainty learning. Second, we incorporate global-level uncertainty supervision by injecting the predicted uncertainty into the sof tmax scale during training. This adaptive scaling mechanism adjusts the sharpness of the decision boundary according to sample difficulty, providing global guidance. Third, we redesign the uncertainty estimation module by integrating a Transformer encoder with multi-view self-attention, enabling the model to capture rich local and long-range temporal dependencies. Comprehensive experiments demonstrate that U3-xi is model-agnostic and can be seamlessly applied to various speaker encoders. In particular, when applied to ECAPA-TDNN, it achieves 21.1% and 15.57% relative improvements on the VoxCeleb1 test sets in terms of EER and minDCF, respectively.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。