构建首个大规模皮肤色调数据集,提升人脸识别公平性评估能力
Large-Scale Dataset and Benchmark for Skin Tone Classification in the Wild
- 用10级MST标准标注4.2万张人脸图像,解决数据粒度不足问题
- 深度模型接近人工标注准确率,传统方法近乎随机分类
- 提出SkinToneNet模型,可跨数据集评估公平性,适合算法审计场景
深度学习模型常继承训练数据中的偏见。尽管性别与族裔公平性研究较成熟,但细粒度皮肤色调分析仍受限于缺乏精细标注的数据集。现有方法多依赖医学6级Fitzpatrick量表(缺乏视觉代表性),或使用小规模私有数据集(不可复现),或依赖传统计算机视觉流程,少数采用深度学习。这些方法普遍存在训练测试泄露、数据分布不均等问题,且受限于数据规模或缺失。本文提出完整皮肤色调公平性框架:首先构建STW数据集,包含42,313张图像、3,564名个体,采用10级MST量表标注;其次基准对比经典计算机视觉(SkinToneCCV)与深度学习方法,发现经典模型表现接近随机,而深度模型达到近标注者水平;最后提出SkinToneNet——微调的ViT模型,在跨域数据上实现最先进的泛化能力,可用于对CelebA、VGGFace2等公开数据集进行可靠公平性审计。本工作在皮肤色调分类与公平性评估方面达到当前最佳水平。代码与数据即将发布。
原文摘要 · Abstract (English)
Deep learning models often inherit biases from their training data. While fairness across gender and ethnicity is well-studied, fine-grained skin tone analysis remains a challenge due to the lack of granular, annotated datasets. Existing methods often rely on the medical 6-tone Fitzpatrick scale, which lacks visual representativeness, or use small, private datasets that prevent reproducibility, or often rely on classic computer vision pipelines, with a few using deep learning. They overlook issues like train-test leakage and dataset imbalance, and are limited by small or unavailable datasets. In this work, we present a comprehensive framework for skin tone fairness. First, we introduce the STW, a large-scale, open-access dataset comprising 42,313 images from 3,564 individuals, labeled using the 10-tone MST scale. Second, we benchmark both Classic Computer Vision (SkinToneCCV) and Deep Learning approaches, demonstrating that classic models provide near-random results, while deep learning reaches nearly annotator accuracy. Finally, we propose SkinToneNet, a fine-tuned ViT that achieves state-of-the-art generalization on out-of-domain data, which enables reliable fairness auditing of public datasets like CelebA and VGGFace2. This work provides state-of-the-art results in skin tone classification and fairness assessment. Code and data available soon
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。