DNA鉴定软件审计标准形同虚设，实操中难保公正
Compliant But Unsatisfactory: The Gap Between Auditing Standards and Practices for Probabilistic Genotyping Software
- 通过分析标准ASB 018和5份审计报告，发现理想与实践脱节
- 合规审计可不设定软件使用限制，即使已发现缺陷
- 标准术语模糊、要求含混，导致执行效果大打折扣
AI治理日益依赖审计标准——即经共识确立的审计规范。然而设计不良的标准可能掩盖并赋予低效系统正当性。本文以美国司法体系日益依赖的概率基因分型软件审计标准ASB 018为案例，通过对其及五份审计报告的定性分析，揭示标准预期目标与实际审计行为之间存在诸多差距。例如，ASB 018期望合规审计能基于观测到的故障限制软件使用范围，但实际审计可完全合规却不设立此类边界。这些差距源于标准中语言模糊、术语未定义等设计缺陷。研究最后提出改进审计标准设计与评估其有效性的建议。
原文摘要 · Abstract (English)
AI governance efforts increasingly rely on audit standards: agreed-upon practices for conducting audits. However, poorly designed standards can hide and lend credibility to inadequate systems. We explore how an audit standard's design influences its effectiveness through a case study of ASB 018, a standard for auditing probabilistic genotyping software -- software that the U.S. criminal legal system increasingly uses to analyze DNA samples. Through qualitative analysis of ASB 018 and five audit reports, we identify numerous gaps between the standard's desired outcomes and the auditing practices it enables. For instance, ASB 018 envisions that compliant audits establish restrictions on software use based on observed failures. However, audits can comply without establishing such boundaries. We connect these gaps to the design of the standard's requirements such as vague language and undefined terms. We conclude with recommendations for designing audit standards and evaluating their effectiveness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。