AI医生虽火，实则仍需人类把关，背后有三大矛盾
The Doctor Will (Still) See You Now: On the Structural Limits of Agentic AI in Healthcare
- 通过20位医疗从业者访谈，发现AI自主性概念模糊
- 商业宣传承诺高自主，实际运行几乎全靠人工监督
- 现有评估重技术指标，忽视医疗系统安全与责任分配
在医疗领域，代理型人工智能（AI）系统被广泛宣传具备自主行动能力，但现实中由于安全、监管和责任限制，其临床决策仍需近乎完全的人类监督。尽管市场热情高涨，但这些系统的概念基础与问责机制仍缺乏深入审视。本文基于对20位利益相关者（包括开发者、实施者和使用者）的定性访谈，识别出三大相互强化的矛盾：对‘代理’定义的概念碎片化；商业承诺与实际操作之间存在的自主性悖论；以及评估体系忽视社会技术安全性，仅关注技术指标。我们指出，代理型AI是多方力量角力的场域，技术愿景、商业动机与临床约束在此交汇，直接影响患者安全与责任归属。
原文摘要 · Abstract (English)
Across healthcare, agentic artificial intelligence (AI) systems are increasingly promoted as capable of autonomous action, yet in practice they currently operate under near-total human oversight due to safety, regulatory, and liability constraints that make autonomous clinical reasoning infeasible in high-stakes environments. While market enthusiasm suggests a revolution in healthcare agents, the conceptual assumptions and accountability structures shaping these systems remain underexamined. We present a qualitative study based on interviews with 20 stakeholders, including developers, implementers, and end users. Our analysis identifies three mutually reinforcing tensions: conceptual fragmentation regarding the definition of `agentic'; an autonomy contradiction where commercial promises exceed operational reality; and an evaluation blind spot that prioritizes technical benchmarks over sociotechnical safety. We argue that agentic {AI} functions as a site of contested meaning-making where technical aspirations, commercial incentives, and clinical constraints intersect, carrying material consequences for patient safety and the distribution of blame.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。