现有生成式AI公平性测试无法满足监管要求，易放行实际歧视的模型。
Towards Effective Discrimination Testing for Generative AI
- 连接法律与技术文献，揭示评估方法与监管目标的错位
- 四个案例显示：现有测试在复杂场景下会遗漏真实歧视行为
- 提出改进方案，提升未来部署中公平性评估的可靠性
生成式AI（GenAI）模型带来了针对歧视行为监管的新挑战。本文指出，当前GenAI公平性研究尚未应对这些挑战，现有偏差评估方法与监管目标之间仍存在显著差距，导致无效监管，可能允许表面公平但实际具有歧视性的GenAI系统被部署。为解决此问题，我们整合法律与技术文献中关于GenAI偏差评估的内容，识别出关键错位点。通过四个案例研究，我们展示这种评估方法与监管目标之间的不一致，在自适应或复杂环境中可能导致歧视性结果。我们提出实用建议，以改进歧视性测试，使其更契合监管目标，提升未来部署中公平性评估的可靠性。
原文摘要 · Abstract (English)
Generative AI (GenAI) models present new challenges in regulating against discriminatory behavior. In this paper, we argue that GenAI fairness research still has not met these challenges; instead, a significant gap remains between existing bias assessment methods and regulatory goals. This leads to ineffective regulation that can allow deployment of reportedly fair, yet actually discriminatory, GenAI systems. Towards remedying this problem, we connect the legal and technical literature around GenAI bias evaluation and identify areas of misalignment. Through four case studies, we demonstrate how this misalignment between fairness testing techniques and regulatory goals can result in discriminatory outcomes in real-world deployments, especially in adaptive or complex environments. We offer practical recommendations for improving discrimination testing to better align with regulatory goals and enhance the reliability of fairness assessments in future deployments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。