arXiv:2411.00831cs.CVcs.AI2024-11被引 1

通过保留显著区域增强图像多样性,提升模型公平性。

Saliency-Based diversity and fairness Metric and FaceKeepOriginalAugment: A Novel Approach for Enhancing Fairness and Diversity

  • 在图像非显著区保留显著特征,实现更智能的增强。
  • 在多数据集上降低性别偏差,提升模型公平性。
  • 提出新度量指标,同时评估多样性与公平性。

数据增强已成为提升计算机视觉性能的关键手段,KeepOriginalAugment方法因其能智能地在不显著区域融入显著区域信息而脱颖而出,实现对显著与非显著区域的双重增强。尽管该方法在图像分类中表现优异,其在缓解模型偏见方面的潜力尚未被探索。本文提出改进方法FaceKeepOriginalAugment,针对地理、性别及刻板印象等偏见问题,在多个数据集(包括FFHQ、WIKI、IMDB、LFW、UTK Faces和Diverse Dataset)上进行研究。通过调整显著区域位置与视角交换策略,结合图像相似度分数(ISS)量化数据多样性,并利用图像-图像关联分数(IIAS)在CNN和视觉变换器(ViTs)中评估性别偏见缓解效果。实验表明,该方法有效降低性别偏差,提升整体公平性。此外,本文提出一种新的‘基于显著性的多样性与公平性度量’,可同时衡量多样性与公平性,并处理各类数据集中的不平衡问题。

原文摘要 · Abstract (English)

Data augmentation has become a pivotal tool in enhancing the performance of computer vision tasks, with the KeepOriginalAugment method emerging as a standout technique for its intelligent incorporation of salient regions within less prominent areas, enabling augmentation in both regions. Despite its success in image classification, its potential in addressing biases remains unexplored. In this study, we introduce an extension of the KeepOriginalAugment method, termed FaceKeepOriginalAugment, which explores various debiasing aspects-geographical, gender, and stereotypical biases-in computer vision models. By maintaining a delicate balance between data diversity and information preservation, our approach empowers models to exploit both diverse salient and non-salient regions, thereby fostering increased diversity and debiasing effects. We investigate multiple strategies for determining the placement of the salient region and swapping perspectives to decide which part undergoes augmentation. Leveraging the Image Similarity Score (ISS), we quantify dataset diversity across a range of datasets, including Flickr Faces HQ (FFHQ), WIKI, IMDB, Labelled Faces in the Wild (LFW), UTK Faces, and Diverse Dataset. We evaluate the effectiveness of FaceKeepOriginalAugment in mitigating gender bias across CEO, Engineer, Nurse, and School Teacher datasets, utilizing the Image-Image Association Score (IIAS) in convolutional neural networks (CNNs) and vision transformers (ViTs). Our findings shows the efficacy of FaceKeepOriginalAugment in promoting fairness and inclusivity within computer vision models, demonstrated by reduced gender bias and enhanced overall fairness. Additionally, we introduce a novel metric, Saliency-Based Diversity and Fairness Metric, which quantifies both diversity and fairness while handling data imbalance across various datasets.

数据增强模型公平性图像多样性性别偏见

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。