自动化纽约市招聘AI偏见检测，发现合规关键挑战
What we learned while automating bias detection in AI hiring systems for compliance with NYC Local Law 144
- 开发工具ITACA_144，适配纽约本地法律要求
- 分析多份审计报告，识别数据与指标缺陷
- 揭示偏见检测中的可操作性难题，供政策参考
自2023年7月5日起，纽约市本地法律144要求雇主对用于招聘的自动化就业决策工具（AEDTs）进行独立偏见审计。该法律列出了必须执行的最低限度偏见测试。我们近期收集并分析了多项依此法律开展的审计，提炼出最佳实践，并开发了软件工具以简化企业合规流程。该工具ITACA_144将我们的通用偏见审计框架定制化以满足本地法律的具体要求。在自动化合规过程中，我们识别出若干关键挑战，这些挑战需引起重视，以确保人工智能偏见监管和审计方法既有效又可行。本文总结了自动化遵守纽约市法律144所获得的洞见，旨在支持其他城市和州制定类似法规，同时回应纽约框架的局限性。讨论重点包括数据要求、人口包容性、影响比率、有效偏见度量指标及数据可靠性等关键领域。
原文摘要 · Abstract (English)
Since July 5, 2023, New York City's Local Law 144 requires employers to conduct independent bias audits for any automated employment decision tools (AEDTs) used in hiring processes. The law outlines a minimum set of bias tests that AI developers and implementers must perform to ensure compliance. Over the past few months, we have collected and analyzed audits conducted under this law, identified best practices, and developed a software tool to streamline employer compliance. Our tool, ITACA_144, tailors our broader bias auditing framework to meet the specific requirements of Local Law 144. While automating these legal mandates, we identified several critical challenges that merit attention to ensure AI bias regulations and audit methodologies are both effective and practical. This document presents the insights gained from automating compliance with NYC Local Law 144. It aims to support other cities and states in crafting similar legislation while addressing the limitations of the NYC framework. The discussion focuses on key areas including data requirements, demographic inclusiveness, impact ratios, effective bias, metrics, and data reliability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。