新基准Genres揭示多模态模型在人际关系中的隐性性别偏见
From Individuals to Interactions: Benchmarking Gender Bias in Multimodal Large Language Models from the Lens of Social Relationship
- 构建双角色互动叙事任务,从社会关系视角评估偏见
- 发现闭源与开源模型均存在情境敏感的性别刻板印象
- 适合关注伦理对齐与社会影响的AI研究者使用
多模态大语言模型在跨视觉与文本任务中表现优异,但其可能内化并放大性别偏见的问题日益突出,尤其在社交敏感场景中。现有评估大多聚焦单个实体,忽视人际互动中潜藏的偏见。本文提出Genres——首个从社会关系视角出发的多模态模型性别偏见评测基准,通过双角色人物设定与叙事生成任务,捕捉丰富的人际动态,支持多维度细粒度偏见分析。在开放与闭源模型上的实验显示,偏见在双人互动中显著显现,且具有情境依赖性,而单角色测试无法察觉。结果表明,需建立关系感知型评测体系以识别交互驱动的细微偏见,并为未来缓解策略提供依据。
原文摘要 · Abstract (English)
Multimodal large language models (MLLMs) have shown impressive capabilities across tasks involving both visual and textual modalities. However, growing concerns remain about their potential to encode and amplify gender bias, particularly in socially sensitive applications. Existing benchmarks predominantly evaluate bias in isolated scenarios, overlooking how bias may emerge subtly through interpersonal interactions. We fill this gap by going beyond single-entity evaluation and instead focusing on a deeper examination of relational and contextual gender bias in dual-individual interactions. We introduce Genres, a novel benchmark designed to evaluate gender bias in MLLMs through the lens of social relationships in generated narratives. Genres assesses gender bias through a dual-character profile and narrative generation task that captures rich interpersonal dynamics and supports a fine-grained bias evaluation suite across multiple dimensions. Experiments on both open- and closed-source MLLMs reveal persistent, context-sensitive gender biases that are not evident in single-character settings. Our findings underscore the importance of relationship-aware benchmarks for diagnosing subtle, interaction-driven gender bias in MLLMs and provide actionable insights for future bias mitigation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。