arXiv:2511.03106cs.AI2025-11被引 3

为通用大模型设计基于能力的监控,提升医疗场景下的安全性和适应性。

Large language models require a new form of oversight: capability-based monitoring

  • 以模型能力为核心组织监控,而非单一任务评估
  • 可跨任务发现系统性缺陷和罕见错误
  • 适合医疗AI开发者与监管机构参考落地

大语言模型在医疗领域的快速应用引发了对其监管的关注。现有基于传统机器学习的监控方法以任务为中心,依赖于数据漂移导致性能下降的假设。然而,大语言模型并非针对特定人群或任务训练,因此无法假设其会因群体变化而必然退化。为此,我们提出一种新的通用监控范式——能力基础监控。该方法基于大语言模型作为通用系统、其内部能力(如摘要、推理、翻译、安全防护)在多个下游任务中复用的特性,不再独立评估每个任务,而是围绕共享能力组织监控,从而实现对系统性弱点、长尾错误及新兴行为的跨任务检测。本文还讨论了开发者、组织管理者及专业协会在实施该方法时需考虑的关键因素。最终,能力基础监控将为医疗领域中大语言模型及未来通用人工智能模型的安全、动态与协作式监控提供可扩展的基础。

原文摘要 · Abstract (English)

The rapid adoption of large language models (LLMs) in healthcare has been accompanied by scrutiny of their oversight. Existing monitoring approaches, inherited from traditional machine learning (ML), are task-based and founded on assumed performance degradation arising from dataset drift. In contrast, with LLMs, inevitable model degradation due to changes in populations compared to the training dataset cannot be assumed, because LLMs were not trained for any specific task in any given population. We therefore propose a new organizing principle guiding generalist LLM monitoring that is scalable and grounded in how these models are developed and used in practice: capability-based monitoring. Capability-based monitoring is motivated by the fact that LLMs are generalist systems whose overlapping internal capabilities are reused across numerous downstream tasks. Instead of evaluating each downstream task independently, this approach organizes monitoring around shared model capabilities, such as summarization, reasoning, translation, or safety guardrails, in order to enable cross-task detection of systemic weaknesses, long-tail errors, and emergent behaviors that task-based monitoring may miss. We describe considerations for developers, organizational leaders, and professional societies for implementing a capability-based monitoring approach. Ultimately, capability-based monitoring will provide a scalable foundation for safe, adaptive, and collaborative monitoring of LLMs and future generalist artificial intelligence models in healthcare.

大模型监控医疗AI能力评估

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。