arXiv:2410.14070cs.CVcs.AI2024-10中稿 · Image Signal and V…被引 10

用注意力区域增强数据,减轻人脸识别中的性别与地域偏见。

FaceSaliencyAug: Mitigating Geographic, Gender and Stereotypical Biases via Saliency-Based Data Augmentation

  • 基于人脸显著区域生成掩码,随机遮蔽并恢复图像以增强多样性。
  • 在5个数据集上提升图像相似度指标,降低性别关联偏差评分。
  • 适用于需公平性的视觉模型训练,尤其关注性别与文化偏见场景。

计算机视觉模型中的地理、性别和刻板印象偏见严重影响其性能与公平性。本文提出FaceSaliencyAug方法,针对卷积神经网络(CNNs)和视觉变换器(ViTs)中的性别偏见问题,利用人脸显著区域检测结果,从预定义搜索空间中随机选取掩码应用于人脸显著区域,并恢复原始图像。该增强策略提升了数据多样性,从而改善模型表现并减少偏见。我们通过图像相似度得分(ISS)在五个数据集(FFHQ、WIKI、IMDB、LFW、UTK Faces、Diverse Dataset)上量化数据多样性,结果显示该方法在ISS-intra和ISS-inter指标上均表现更优。此外,在CEO、Engineer、Nurse、School Teacher四个职业数据集上,采用图像-图像关联得分(IIAS)评估性别偏见,实验表明该方法显著降低两种模型的性别偏见,验证了其在提升视觉模型公平性与包容性方面的有效性。

原文摘要 · Abstract (English)

Geographical, gender and stereotypical biases in computer vision models pose significant challenges to their performance and fairness. {In this study, we present an approach named FaceSaliencyAug aimed at addressing the gender bias in} {Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs). Leveraging the salient regions} { of faces detected by saliency, the propose approach mitigates geographical and stereotypical biases } {in the datasets. FaceSaliencyAug} randomly selects masks from a predefined search space and applies them to the salient region of face images, subsequently restoring the original image with masked salient region. {The proposed} augmentation strategy enhances data diversity, thereby improving model performance and debiasing effects. We quantify dataset diversity using Image Similarity Score (ISS) across five datasets, including Flickr Faces HQ (FFHQ), WIKI, IMDB, Labelled Faces in the Wild (LFW), UTK Faces, and Diverse Dataset. The proposed approach demonstrates superior diversity metrics, as evaluated by ISS-intra and ISS-inter algorithms. Furthermore, we evaluate the effectiveness of our approach in mitigating gender bias on CEO, Engineer, Nurse, and School Teacher datasets. We use the Image-Image Association Score (IIAS) to measure gender bias in these occupations. Our experiments reveal a reduction in gender bias for both CNNs and ViTs, indicating the efficacy of our method in promoting fairness and inclusivity in computer vision models.

人脸偏见数据增强公平性显著性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。