生成模型公平性问题本质是评估标准缺失，需统一评测规范
Position: Fairness Failure in Generative Models is an Evaluation Problem
- 提出公平性卡片，明确评估细节如提示模板与反事实协议
- 当前研究结果不可比，部署决策缺乏可靠依据
- 适合关注模型伦理与可复现评估的研究者和开发者
尽管过去十年生成模型取得突破性进展，但其公平性缺陷——加剧社会不平等并伤害边缘群体——仍未能有效解决。本文认为，生成模型的公平性失败本质上是评估问题：现有研究结果难以比较，也难以用于实际部署决策。文章诊断了当前实践中反复出现的实证与概念性错误，并呼吁从临时性偏见检测转向标准化、专为生成模型设计的评估体系。为此提出‘公平性卡片’作为最小化报告模板，显式披露评估选择（如提示族、反事实协议、度量指标、拒绝处理方式），以提升可复现性、可比性和问责性。最后提出多项改进建议，推动评估范式转型。项目主页见：https://mariiavladimirova.github.io/fairness-cards。
原文摘要 · Abstract (English)
Despite groundbreaking advancements in generative models during the last decade, concerns about their lack of fairness, reinforcing societal inequalities and harming marginalized groups, remain under-addressed and difficult to act upon. This position paper argues that fairness failures in generative models, albeit driven by multiple factors, are ultimately stemming from an evaluation problem: fairness findings are rarely comparable across papers or actionable for deployment decisions. This paper diagnoses recurring empirical and conceptual failure modes in current practice and motivates a shift from ad-hoc bias checks to standardized, generative-specific evaluation. We propose Fairness Cards as a minimal reporting artifact that makes evaluation choices explicit (prompt families, counterfactual protocols, metrics, and refusal handling) enabling reproducibility, comparability, and accountability. We conclude with additional recommendations towards a paradigm shift in evaluation standards. Our project page can be found at https://mariiavladimirova.github.io/fairness-cards .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。