无需人工标注，快速评估人脸识别准确率与偏见。
A Rapid Test for Accuracy and Bias of Face Recognition Technology
- 利用模型嵌入表示和近似标签，实现无标注快速验证。
- 在小数据集上准确估计精度与排名，显著降低成本。
- 首次公开五家云服务对比，揭示亚裔女性识别偏差。
衡量人脸识别（FR）系统的准确性对提升性能和确保负责任使用至关重要。传统方法依赖大型标注数据集，成本高且难获取。本文提出一种新型1:1人脸识别验证方法，可在无需人工标注的情况下，仅基于近似标签（如网络搜索结果）快速评估系统表现。不同于以往训练集标签清洗方法，本方法利用待测模型的嵌入表示，在较小测试集上实现高精度估计。该方法可可靠评估人脸识别准确率与排名，大幅减少人工标注的时间与成本。我们还构建了首个包含五家主流云服务的人脸识别公开基准，发现亚裔女性群体存在明显识别准确率下降现象。该快速测试工具已开源，网址为 https://github.com/caltechvisionlab/frt-rapid-test，助力技术透明化与责任化应用。
原文摘要 · Abstract (English)
Measuring the accuracy of face recognition (FR) systems is essential for improving performance and ensuring responsible use. Accuracy is typically estimated using large annotated datasets, which are costly and difficult to obtain. We propose a novel method for 1:1 face verification that benchmarks FR systems quickly and without manual annotation, starting from approximate labels (e.g., from web search results). Unlike previous methods for training set label cleaning, ours leverages the embedding representation of the models being evaluated, achieving high accuracy in smaller-sized test datasets. Our approach reliably estimates FR accuracy and ranking, significantly reducing the time and cost of manual labeling. We also introduce the first public benchmark of five FR cloud services, revealing demographic biases, particularly lower accuracy for Asian women. Our rapid test method can democratize FR testing, promoting scrutiny and responsible use of the technology. Our method is provided as a publicly accessible tool at https://github.com/caltechvisionlab/frt-rapid-test
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。