arXiv:2508.11262cs.CVcs.AI2025-08被引 5

发现视觉语言模型对职业描述存在性别偏见,男性关联更强。

Vision-Language Models display a strong gender bias

  • 用对比学习构建图像与文本嵌入,量化性别关联强度
  • 6类劳动中男性职业描述平均关联度高出18.7%
  • 提供可复现的偏见评估框架,适合研究公平性者参考

视觉语言模型(VLM)将图像与文本映射到共享表示空间,可用于检索和零样本迁移。然而,这种对齐可能以不易察觉的方式编码并放大社会刻板印象。本研究测试对比型视觉语言编码器在将人脸图像嵌入与描述职业和活动的短语嵌入对齐时是否存在性别关联。我们构建了一个包含220张按感知性别划分的人脸照片的数据集,以及150个涵盖情感劳动、认知劳动、家务劳动、技术劳动、职业角色和体力劳动六类的独特陈述。为每张人脸计算单位范数图像嵌入,为每个陈述计算单位范数文本嵌入,定义陈述层面的关联分数为男性组均值余弦相似度与女性组均值余弦相似度之差,正值表示与男性更强关联,负值表示与女性更强关联。通过在每性别组内重采样生成置信区间,按类别聚合时另做一次陈述层面的重采样,并运行标签互换零模型,估算无性别结构时应有的平均绝对关联水平。最终获得一个陈述级与类别级的性别关联图谱,附带不确定性估计、基础验证及鲁棒的性别偏见评估框架。

原文摘要 · Abstract (English)

Vision-language models (VLM) align images and text in a shared representation space that is useful for retrieval and zero-shot transfer. Yet, this alignment can encode and amplify social stereotypes in subtle ways that are not obvious from standard accuracy metrics. In this study, we test whether the contrastive vision-language encoder exhibits gender-linked associations when it places embeddings of face images near embeddings of short phrases that describe occupations and activities. We assemble a dataset of 220 face photographs split by perceived binary gender and a set of 150 unique statements distributed across six categories covering emotional labor, cognitive labor, domestic labor, technical labor, professional roles, and physical labor. We compute unit-norm image embeddings for every face and unit-norm text embeddings for every statement, then define a statement-level association score as the difference between the mean cosine similarity to the male set and the mean cosine similarity to the female set, where positive values indicate stronger association with the male set and negative values indicate stronger association with the female set. We attach bootstrap confidence intervals by resampling images within each gender group, aggregate by category with a separate bootstrap over statements, and run a label-swap null model that estimates the level of mean absolute association we would expect if no gender structure were present. The outcome is a statement-wise and category-wise map of gender associations in a contrastive vision-language space, accompanied by uncertainty, simple sanity checks, and a robust gender bias evaluation framework.

视觉语言模型性别偏见公平性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。