arXiv:2505.14160cs.CLcs.LG2025-05中稿 · IJCNLP-AACL 2025被引 2

多语言视觉模型反而放大性别种族偏见,低资源语言尤其严重。

Breaking Language Barriers or Reinforcing Bias? A Study of Gender and Racial Disparities in Multilingual Contrastive Vision Language Models

  • 对比四种多语言CLIP模型,发现跨语言权重共享会传递英语偏见
  • 低资源语言中性别偏差最强,高语义性别化语言放大所有偏见
  • 现有评估忽略语言差异,需细粒度的本地化偏见检测

多语言视觉语言模型(VLMs)承诺实现跨语言图文检索,但其社会偏见尚未被充分研究。本文首次系统审计了四种公开的多语言CLIP变体:M-CLIP、NLLB-CLIP、CAPIVARA-CLIP和去偏的SigLIP-2,覆盖十种语言,涵盖资源丰富与稀缺、有无形态性别标记的语言。在零样本设置下,使用平衡的FairFace和PATA刻板印象套件,量化了种族与性别偏见及刻板印象放大效应。出人意料的是,所有模型的性别偏差均强于其英文单语基线。CAPIVARA-CLIP在目标低资源语言中表现出最大偏差;而NLLB-CLIP与SigLIP-2的共享编码器将英语性别刻板印象迁移至无性别标记语言;松耦合编码器则显著避免了这种泄露。尽管SigLIP-2降低了主动性与亲属性偏差,但在图像描述稀疏场景(如祖鲁语)中,仍继承并放大了英语锚点的犯罪关联。高度性别化的语言持续放大各类偏见,而无性别标记语言在跨语言权重共享导入外来偏见时依然脆弱。聚合指标掩盖了语言特定热点,凸显未来多语言VLM研究亟需细粒度、语言感知的偏见评估。

原文摘要 · Abstract (English)

Multilingual vision-language models (VLMs) promise universal image-text retrieval, yet their social biases remain underexplored. We perform the first systematic audit of four public multilingual CLIP variants: M-CLIP, NLLB-CLIP, CAPIVARA-CLIP, and the debiased SigLIP-2, covering ten languages that differ in resource availability and morphological gender marking. Using balanced subsets of FairFace and the PATA stereotype suite in a zero-shot setting, we quantify race and gender bias and measure stereotype amplification. Contrary to the intuition that multilinguality mitigates bias, every model exhibits stronger gender skew than its English-only baseline. CAPIVARA-CLIP shows its largest biases precisely in the low-resource languages it targets, while the shared encoder of NLLB-CLIP and SigLIP-2 transfers English gender stereotypes into gender-neutral languages; loosely coupled encoders largely avoid this leakage. Although SigLIP-2 reduces agency and communion skews, it inherits -- and in caption-sparse contexts (e.g., Xhosa) amplifies -- the English anchor's crime associations. Highly gendered languages consistently magnify all bias types, yet gender-neutral languages remain vulnerable whenever cross-lingual weight sharing imports foreign stereotypes. Aggregated metrics thus mask language-specific hot spots, underscoring the need for fine-grained, language-aware bias evaluation in future multilingual VLM research.

多语言模型偏见检测视觉语言公平性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。