arXiv:2503.05093cs.CV2025-03被引 1

视觉语言模型对性别和种族的刻板印象不仅限于标签关联,还体现在叙事单一化上。

Visual Cues of Gender and Race are Associated with Stereotyping in Vision-Language Models

  • 用典型面部图像测试模型,发现女性叙事更单一,越符合性别典型者越明显
  • 白人比黑人被描述得更同质,但种族典型性不影响叙事单一程度
  • 模型普遍将黑人与篮球关联,其他刻板印象因模型而异,提示现有缓解策略不足

当前视觉语言模型(VLMs)偏见研究存在局限:仅关注特质关联,忽视其他刻板表现形式;只在预设情境下分析;将性别、种族视为二元分类。本研究使用具有不同典型性的标准人脸图像,在开放语境下测试四个VLMs的特质关联与同质性偏见。结果发现,模型对女性生成的叙事更趋一致,且越符合性别典型外观的人,叙事越单一;对白人美国人的描述也比黑人更统一。然而,种族典型性并未增强这种同质性。在特质关联方面,所有模型均将黑人美国人与篮球相关联,其他关联(如艺术、医疗、外貌)则因模型而异。这些结果表明,模型刻板印象的表现形式远超简单群体归属,常规偏见缓解手段可能不足以应对此类问题,同质性偏见即使在无明显特质关联时仍持续存在。

原文摘要 · Abstract (English)

Current research on bias in Vision Language Models (VLMs) has important limitations: it is focused exclusively on trait associations while ignoring other forms of stereotyping, it examines specific contexts where biases are expected to appear, and it conceptualizes social categories like race and gender as binary, ignoring the multifaceted nature of these identities. Using standardized facial images that vary in prototypicality, we test four VLMs for both trait associations and homogeneity bias in open-ended contexts. We find that VLMs consistently generate more uniform stories for women compared to men, with people who are more gender prototypical in appearance being represented more uniformly. By contrast, VLMs represent White Americans more uniformly than Black Americans. Unlike with gender prototypicality, race prototypicality was not related to stronger uniformity. In terms of trait associations, we find limited evidence of stereotyping-Black Americans were consistently linked with basketball across all models, while other racial associations (i.e., art, healthcare, appearance) varied by specific VLM. These findings demonstrate that VLM stereotyping manifests in ways that go beyond simple group membership, suggesting that conventional bias mitigation strategies may be insufficient to address VLM stereotyping and that homogeneity bias persists even when trait associations are less apparent in model outputs.

视觉语言模型刻板印象偏见检测同质性偏见

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。