研究职业名称性别化对招聘匹配系统的影响，发现多语言模型普遍存在性别偏见。
Measuring Gender Bias in Job Title Matching for Grammatical Gender Languages
- 用排序偏差重叠度衡量性别偏见，控制性别因素评估匹配结果
- 构建四种语法性别语言的测试集，含男女形式职业名称及匹配标注
- 实证多个现成多语言模型均存在不同程度性别偏见，可作基准参考
本研究为分析职业名称中显式语法性别分配如何影响自动职位匹配系统的结果奠定了基础。我们提出使用控制性别的排序比较指标（如RBO）来评估职位匹配系统中的性别偏见。为此，我们在四种具有语法性别的语言中生成并共享了职位匹配任务的测试集，包含男性和女性形式的职业名称，并由人工标注性别和匹配相关性。利用这些新测试集和所提方法，评估了多个现成的多语言模型，发现所有模型均表现出不同程度的性别偏见，为后续研究提供了基准。
原文摘要 · Abstract (English)
This work sets the ground for studying how explicit grammatical gender assignment in job titles can affect the results of automatic job ranking systems. We propose the usage of metrics for ranking comparison controlling for gender to evaluate gender bias in job title ranking systems, in particular RBO (Rank-Biased Overlap). We generate and share test sets for a job title matching task in four grammatical gender languages, including occupations in masculine and feminine form and annotated by gender and matching relevance. We use the new test sets and the proposed methodology to evaluate the gender bias of several out-of-the-box multilingual models to set as baselines, showing that all of them exhibit varying degrees of gender bias.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。