为医疗AI设计统一日志协议,实现使用全过程可追溯。
A global log for medical AI
- 提出MedLog日志协议,记录AI与人、系统交互的九类关键信息。
- 在四国部署中发现模型性能下降及诊疗行为变化等真实影响。
- 支持低资源环境运行,适合医疗AI监管与持续优化场景。
现代计算机系统依赖syslog协议记录跨异构基础设施的关键事件,但医学领域快速发展的AI系统尚无类似标准。随着AI工具大规模应用,缺乏对模型使用时间、地点、使用者及对象的标准化记录,难以评估实际效果、检测不良事件或识别偏差与数据漂移。本文提出MedLog,一种面向医疗AI的事件级日志协议。每次AI与人类、其他算法或自动化流程交互时,均生成一条包含九个核心字段(头信息、模型、用户、目标、输入、产物、输出、结果、反馈)的记录。我们在美国、瑞士和越南的四个部署中应用:重症监护室恶化预测、可穿戴设备监测破伤风进展、自动败血症质量报告及患者到诊预测。日志记录了模型行为、工作流交互及下游结果,包括患者到诊预测中恶劣天气导致的性能下降,以及重症监护预警后实验室检查增加的现象。MedLog通过风险采样、生命周期保留策略和写后缓存降低数据开销,支持复杂、自主或多阶段工作流的详细追踪,为医疗AI的持续监控、审计与改进奠定基础。
原文摘要 · Abstract (English)
Modern computer systems rely on syslog, a universal protocol that records critical events across heterogeneous infrastructure. Medicine's rapidly growing AI stack has no equivalent. As medicine deploys AI tools at scale, there is no standard way to record how, when, by whom, and for whom these models are used. Without such records, it is difficult to measure real-world performance and outcomes, detect adverse events, or identify bias and dataset drift. Here we introduce MedLog, a protocol for event-level logging of medical AI. Each time an AI model interacts with a human, another algorithm, or an automated workflow, MedLog creates a record. Each record contains nine core fields: header, model, user, target, inputs, artifacts, outputs, outcomes, and feedback. We apply MedLog across four deployments in the US, Switzerland, and Vietnam: ICU deterioration prediction, tetanus progression monitoring from wearable signals, automated sepsis quality reporting, and patient attendance prediction. MedLog records capture model behavior, workflow interactions, and downstream outcomes, including AI performance degradation during severe weather events in patient attendance prediction and increased laboratory testing after ICU deterioration alerts. MedLog limits the data footprint through risk-based sampling, lifecycle-aware retention policies, and write-behind caching, enabling deployment in low-resource settings. It also supports detailed traces for complex, agentic, or multi-stage workflows, creating a foundation for continuous monitoring, auditing, and improvement of medical AI.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。