构建可衡量的AI治理多元性指标,助力公众监督技术发展
Measuring What Matters: The AI Pluralism Index
- 从参与治理、多样性、透明度和问责四方面量化AI治理多元性
- 基于公开资料与独立评估,提供可验证的下限得分与覆盖范围报告
- 适合政策制定者、采购方及公众用于比较不同AI系统治理水平
人工智能系统日益主导知识传播、沟通与决策。然而其研发与治理仍集中在少数企业与国家,可能隐含狭隘利益,削弱公众自主性。尽管语言、视觉和编码能力基准常见,但针对治理多元性的公开可审计度量仍极为稀缺。本文定义‘AI多元性’为受影响利益相关方在目标设定、数据实践、安全机制与部署中的话语权程度,并提出AI多元性指数(AIPI),一个透明、基于证据的评估工具,涵盖参与式治理、包容性与多样性、透明度、问责性四大支柱。AIPI通过编码公开文档与独立评估中的可验证实践,明确处理‘未知’证据,报告最低得分(‘有证据’)与仅知得分,并标注覆盖率。研究建立了可复现的分析流程,整合结构化网页与代码库分析、外部评估与专家访谈,并通过评分者一致性、覆盖率报告、跨指数相关性与敏感性分析验证可靠性。协议、编码手册、打分脚本与证据图谱均开放发布,支持版本控制与公众裁决。报告了试点提供商结果,并将AIPI置于相邻透明度、安全与治理框架中定位。该指数旨在引导激励向多元治理倾斜,为政策制定者、采购方与公众提供可比证据。
原文摘要 · Abstract (English)
Artificial intelligence systems increasingly mediate knowledge, communication, and decision making. Development and governance remain concentrated within a small set of firms and states, raising concerns that technologies may encode narrow interests and limit public agency. Capability benchmarks for language, vision, and coding are common, yet public, auditable measures of pluralistic governance are rare. We define AI pluralism as the degree to which affected stakeholders can shape objectives, data practices, safeguards, and deployment. We present the AI Pluralism Index (AIPI), a transparent, evidence-based instrument that evaluates producers and system families across four pillars: participatory governance, inclusivity and diversity, transparency, and accountability. AIPI codes verifiable practices from public artifacts and independent evaluations, explicitly handling "Unknown" evidence to report both lower-bound ("evidence") and known-only scores with coverage. We formalize the measurement model; implement a reproducible pipeline that integrates structured web and repository analysis, external assessments, and expert interviews; and assess reliability with inter-rater agreement, coverage reporting, cross-index correlations, and sensitivity analysis. The protocol, codebook, scoring scripts, and evidence graph are maintained openly with versioned releases and a public adjudication process. We report pilot provider results and situate AIPI relative to adjacent transparency, safety, and governance frameworks. The index aims to steer incentives toward pluralistic practice and to equip policymakers, procurers, and the public with comparable evidence.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。