为AI影响评估选对指标，关键在于说清背后的伦理理念
Measuring the right thing: justifying metrics in AI impact assessments
- 先明确伦理概念（如罗尔斯公平或团结公平），再匹配相应指标
- 不同公平概念能帮助解释为何选择特定指标而非其他
- 适合关注AI伦理、政策制定与评估方法的研究者
AI影响评估的有效性取决于所用指标的质量。因此，必须能够合理解释评估中指标的选择，特别是对于难以量化的伦理与社会价值。本文提出两步法：首先明确伦理概念（如罗尔斯式公平或团结公平），其次将指标与该概念相匹配。两个步骤均需独立论证：概念需根据其与公平等功能的契合度进行评判，而概念本身可借助概念工程工具加以构建。通过对比多个公平性指标，我们发现概念提供的额外内涵有助于合理化特定指标的选择。因此，建议影响评估不仅清晰说明所用指标，也阐明其背后支撑的伦理概念。
原文摘要 · Abstract (English)
AI Impact Assessments are only as good as the measures used to assess the impact of these systems. It is therefore paramount that we can justify our choice of metrics in these assessments, especially for difficult to quantify ethical and social values. We present a two-step approach to ensure metrics are properly motivated. First, a conception needs to be spelled out (e.g. Rawlsian fairness or fairness as solidarity) and then a metric can be fitted to that conception. Both steps require separate justifications, as conceptions can be judged on how well they fit with the function of, for example, fairness. We argue that conceptual engineering offers helpful tools for this step. Second, metrics need to be fitted to a conception. We illustrate this process through an examination of competing fairness metrics to illustrate that here the additional content that a conception offers helps us justify the choice for a specific metric. We thus advocate that impact assessments are not only clear on their metrics, but also on the conceptions that motivate those metrics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。