arXiv:2606.20680cs.CVeess.AS2026-06

生物识别验证应以低误匹配率下的性能为准,而非笼统的ROC-AUC。

Beyond ROC-AUC: Operating-Point Performance Reporting for Biometric Verification

论文配图:Beyond ROC-AUC: Operating-Point Performance Reporting for Biometric Verification
图 1 · 摘自论文原文
  • 强调在低误匹配率(如FMR=10^-3)下报告真实匹配率
  • 实测显示全ROC-AUC排名与实际部署性能可能相反
  • 推荐用DET曲线和固定阈值下的错误率作为主报告指标

生物识别系统通常在严格误匹配率预算下运行,仅使用得分范围中极低误匹配率(FMR)的部分。当前标准ISO/IEC 19795-1要求报告特定工作点上的错误率、检测误差权衡(DET)曲线及每个值的置信区间。然而实践中仍普遍使用全区域的ROC-AUC、等错误率(EER)或验证准确率作为单一指标,这些指标不被标准认可。全ROC-AUC对从0到1的整个FMR范围赋予等权重,导致几乎全部权重落在系统从不使用的高误匹配区域,掩盖了低FMR表现,甚至颠倒系统排序。本文重新审视该指南,在人脸、语音、虹膜、指纹四种模态上测试七个预训练匹配器,采用自助法置信区间和配对检验。结果显示:在全ROC-AUC上表现更好的系统,在FMR=10^-3时反而显著更差;例如人脸识别中,FaceNet的全AUC更高,但ArcFace在FMR=10^-3时的真实匹配率(TMR)更高,且差异显著,置信区间无重叠。因此本文重申,应以DET曲线和固定FMR下的FNMR为主报告指标,而保留ROC-AUC与EER作为补充参考。

原文摘要 · Abstract (English)

A biometric verifier is often deployed with a strict false match budget, so only a narrow, low false match rate (FMR) slice of the score range is used. A reporting standard for this setting already exists. ISO/IEC 19795-1 asks for error rates at stated operating points, for the detection error tradeoff (DET) curve as the view of the trade-off between FMR and the false non-match rate (FNMR), and for an interval of uncertainty on every value. In practice, a single area under the receiver operating characteristic curve (ROC-AUC), the equal error rate (EER), or a verification accuracy is still reported as the resolution, which is a threshold-independent summary that the standard does not endorse. The full ROC-AUC averages the true match rate (TMR) with equal weight over the whole FMR range from 0 to 1, so almost all of its weight is placed where the system is never operated; low-FMR behavior can then be hidden, and the order of two systems can even be reversed. The guideline is revisited in this paper and tested against seven pretrained matchers across four modalities, face, voice, iris, and fingerprint, each reported with bootstrap confidence intervals and paired bootstrap tests. A system that looks stronger on full ROC-AUC is shown to be significantly worse at FMR = 10^-3. For face, a higher full AUC was obtained by FaceNet, whereas a higher TMR at FMR = 10^-3 was obtained by ArcFace, and both gaps were significant with non-overlapping intervals. Hence, the DET curve and the FNMR at a fixed FMR are re-iterated in this paper as the primary report, with ROC-AUC and EER retained as supplementary context.

生物识别性能评估验证系统指标优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。