AI自主性提升反削弱人类监督能力,需主动设计支持人机协同。
AI Agents Push Humans Out of the Loop

- 将人类监督需求纳入AI代理设计核心,而非事后补救。
- 长期使用AI导致人类判断力退化,现有系统加剧此问题。
- 提出可操作的设计与组织方案,防止技能衰减。
AI代理的自主性不断增强,带来显著风险。尽管常提议通过人类监督来保障安全,但当前的AI代理设计本身阻碍有效的人类干预,且长期依赖AI会削弱人类所需的认知能力。本文认为,现有开发与部署模式不仅无法支持有效的人类监督,反而促使其退化。为此,应将支持人类监督者的实际目标与认知需求置于与提升AI能力同等重要的地位。结合自动化与人机交互研究,我们提出在设计层面提供支持机制,并建立组织规程,以促进监督者保持批判性判断力,并抵消自动化带来的技能衰退。若不明确支持人类在人机协作中的认知需求,AI系统将持续被动激励其依赖的人类能力退化。
原文摘要 · Abstract (English)
AI agents pose significant risks as they are granted increasing autonomy. A commonly proposed solution is human oversight and keeping a ''human in the loop'', but this is not a simple solution: Not only do current approaches to AI agent design impede effective human oversight, but the cognitive capacities required for it are also themselves degraded by extended use of AI systems. This position paper argues that current approaches to the development and deployment of AI agent systems do not support effective human oversight -- they contribute to its degradation. To address this, a top priority in the advancement of AI agents should be supporting the situated goals and cognitive requirements of effective human oversight, treating the human needs of overseers at the same level of importance as AI agent capability. To put this idea into practice, we connect work on automation and human-computer interaction to AI agent processes, outlining design-level affordances and organizational protocols that (1) support overseers in exercising critical judgement and (2) counteract the skill atrophy that arises from extended use of automation. We urge developers and deployers to adopt these or similar approaches. Without explicit support for the cognitive demands of effective human-agent interaction, AI agent systems will continue to passively incentivize the degradation of the very human skills they rely on.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。