提出AI支持系统如何保护专业领域自主权的设计框架。
Autonomy by Design: Preserving Human Autonomy in AI Decision-Support
- 通过角色定义与失效检测机制,防止AI侵蚀专业判断力
- 实证显示缺乏故障提示会削弱长期专业能力与价值判断
- 适合医疗、金融等需高专业自主性的领域使用
AI系统在专业、技能型及个人活动中日益支持人类决策。尽管已有研究关注AI对人类自主权的总体影响,但其对领域特定自主权——即在特定技能或专长范围内自我掌控行动的能力——的影响仍被忽视。本文分析了AI决策支持系统对两个关键要素的影响:专业能力(在自身领域内做出知情判断的能力)和真实价值形成(建立与领域相关的真正价值观与偏好)。通过跨医疗、金融与教育领域的实证案例,我们揭示了可靠故障指示缺失与无意识价值偏移如何即时且持续地削弱领域特定自主权。为此,我们构建了一个保持自主权的AI支持系统设计框架,提出具体社会技术设计模式,包括明确定义角色、实施失效终止机制以及支持反思性实践,以在利用AI能力的同时维持专业领域内的人类能动性。该框架为开发增强而非削弱人类自主性的专用领域系统提供了具体指导。
原文摘要 · Abstract (English)
AI systems increasingly support human decision-making across domains of professional, skill-based, and personal activity. While previous work has examined how AI might affect human autonomy globally, the effects of AI on domain-specific autonomy -- the capacity for self-governed action within defined realms of skill or expertise -- remain understudied. We analyze how AI decision-support systems affect two key components of domain-specific autonomy: skilled competence (the ability to make informed judgments within one's domain) and authentic value-formation (the capacity to form genuine domain-relevant values and preferences). By engaging with prior investigations and analyzing empirical cases across medical, financial, and educational domains, we demonstrate how the absence of reliable failure indicators and the potential for unconscious value shifts can erode domain-specific autonomy both immediately and over time. We then develop a constructive framework for autonomy-preserving AI support systems. We propose specific socio-technical design patterns -- including careful role specification, implementation of defeater mechanisms, and support for reflective practice -- that can help maintain domain-specific autonomy while leveraging AI capabilities. This framework provides concrete guidance for developing AI systems that enhance rather than diminish human agency within specialized domains of action.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。