语音伪造检测模型部署时阈值迁移会失效，导致大量真实语音被误判。
When EER Hides Deployment Failure: Auditing Threshold Transfer and Unlabeled Score Calibration for Speech Deepfake Detectors
- 用真实数据测试预设阈值下的检测效果，揭示评估误差
- 7种常用校准方法均无法改善阈值迁移后的错误率
- 建议同时报告迁移后的真实误判率，而不仅是理想误差率
语音伪造对抗措施（CMs）几乎全依赖等错误率（EER）进行比较，该指标基于标注测试集上的最优阈值计算。然而实际部署中，阈值需预先固定并应用于未标注目标数据，与训练环境存在巨大差距。本文以ASVspoof 2019 LA数据集上训练的SOTA SSL-AASIST模型为例，其在域内EER为0.21%，但将该阈值迁移到In-the-Wild数据集后，总错误率（HTER）高达39.5%，其中78.7%的真实语音被错误拒绝，尽管其在In-the-Wild上的EER为11.2%。随后测试七种常见无标签测试时校准方法，验证了一个简单命题：任何严格递增的得分变换（如z-norm、温度/偏移校准、嵌入均值对齐）都无法改变EER。实证发现，使用无标签目标数据组的AS-norm校准导致性能崩溃（EER从11.2%升至60.2%），伪标签校准虽在In-the-Wild上将HTER降低38%，但在DF21数据集（伪造比例96%）上退化至50% HTER。所有方法的相对EER改善均不超过1%。建议在报告EER的同时，补充在迁移阈值下的HTER结果。
原文摘要 · Abstract (English)
Speech deepfake countermeasures (CMs) are compared almost exclusively by equal error rate (EER), a metric computed at an oracle threshold chosen on the labeled test set. Deployed CMs enjoy no such oracle: a threshold must be fixed in advance and applied to unlabeled target data. We audit this gap with a frozen state-of-the-art SSL-AASIST detector trained on ASVspoof 2019 LA. While its in-domain EER is 0.21%, transferring its LA-calibrated threshold to the In-the-Wild corpus yields a half total error rate (HTER) of 39.5%, with 78.7% of bona fide speech rejected, even though the In-the-Wild EER (11.2%) appears moderate. We then test whether popular unlabeled test-time corrections close this gap, and first prove a simple proposition: any strictly increasing score transform, including z-norm, temperature/shift calibration, and embedding mean alignment under a frozen linear head, cannot change EER. An audit of seven corrections on In-the-Wild and ASVspoof 2021 DF confirms the proposition empirically and exposes two further failure modes: AS-norm with an unlabeled target cohort collapses (EER 11.2% to 60.2%), and pseudo-label calibration that reduces HTER by 38% relative on In-the-Wild degenerates to 50% HTER on DF21, whose spoof prior is 96%. No audited correction reduces EER by more than 1% relative. We recommend reporting HTER at a transferred threshold alongside EER.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。