发现医疗视觉模型存在职业与人种性别刻板印象
Surgeons Are Indian Males and Speech Therapists Are White Females: Auditing Biases in Vision-Language Models for Healthcare Professionals
- 构建医疗角色分类体系并设计针对性提示测试模型偏见
- 多个模型在不同医护角色中均显示显著人种性别偏差
- 提醒医疗AI应用需警惕偏见，避免影响招聘公平性
视觉语言模型（如CLIP和OpenCLIP）能编码并反映从网络规模数据中学到的医疗职业与人口属性之间的刻板印象。我们提出一种面向医疗场景的评估协议，量化关联偏见并评估其操作风险。方法包括：(i) 构建涵盖临床医生及辅助医疗角色（如外科医生、心脏病专家、牙医、护士、药剂师、技术人员）的分类体系；(ii) 设计专业感知提示集以探测模型行为；(iii) 以均衡人脸数据集为基准对比人口分布偏移。实证发现，多种职业和视觉模型均存在持续的种族与性别偏差。研究强调在医疗等关键领域识别偏见的重要性，因人工智能支持的招聘与人力资源分析可能对公平性、合规性和患者信任产生深远影响。
原文摘要 · Abstract (English)
Vision language models (VLMs), such as CLIP and OpenCLIP, can encode and reflect stereotypical associations between medical professions and demographic attributes learned from web-scale data. We present an evaluation protocol for healthcare settings that quantifies associated biases and assesses their operational risk. Our methodology (i) defines a taxonomy spanning clinicians and allied healthcare roles (e.g., surgeon, cardiologist, dentist, nurse, pharmacist, technician), (ii) curates a profession-aware prompt suite to probe model behavior, and (iii) benchmarks demographic skew against a balanced face corpus. Empirically, we observe consistent demographic biases across multiple roles and vision models. Our work highlights the importance of bias identification in critical domains such as healthcare as AI-enabled hiring and workforce analytics can have downstream implications for equity, compliance, and patient trust.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。