提出动态验证框架,让医疗AI设备持续适应真实临床环境。
Beyond One-Time Validation: A Framework for Adaptive Validation of Prognostic and Diagnostic AI-based Medical Devices
- 引入部署后重复验证与调优机制,应对实际使用中的变化。
- 强调在不同医疗机构间保持性能稳定,提升可靠性。
- 适合关注医疗AI落地与合规的开发者和监管者。
预测性与诊断性基于人工智能的医疗设备在推进医疗进步方面潜力巨大,但其快速发展已超越了适当验证方法的建立。现有方法常无法应对实际部署中的复杂性,难以确保设备在真实临床环境中的有效、持续运行。本文基于近期关于医学AI模型验证的讨论,并借鉴其他领域的验证实践,提出一个结构化、稳健的验证框架,旨在保障设备在不同临床环境中的一致可靠性。文章分析了设备部署后性能下降的主要挑战,尤其关注各医疗机构及运营流程变化的影响。框架强调部署期间需反复进行验证与微调,以应对开发阶段未预见的问题。同时,该框架被置于当前美国和欧盟的监管体系中,凸显其实际可行性与合规相关性。此外,通过一个实际案例展示了框架的潜在优势,并提供评估模型性能的指导,强调临床利益相关方参与验证与调优的重要性。
原文摘要 · Abstract (English)
Prognostic and diagnostic AI-based medical devices hold immense promise for advancing healthcare, yet their rapid development has outpaced the establishment of appropriate validation methods. Existing approaches often fall short in addressing the complexity of practically deploying these devices and ensuring their effective, continued operation in real-world settings. Building on recent discussions around the validation of AI models in medicine and drawing from validation practices in other fields, a framework to address this gap is presented. It offers a structured, robust approach to validation that helps ensure device reliability across differing clinical environments. The primary challenges to device performance upon deployment are discussed while highlighting the impact of changes related to individual healthcare institutions and operational processes. The presented framework emphasizes the importance of repeating validation and fine-tuning during deployment, aiming to mitigate these issues while being adaptable to challenges unforeseen during device development. The framework is also positioned within the current US and EU regulatory landscapes, underscoring its practical viability and relevance considering regulatory requirements. Additionally, a practical example demonstrating potential benefits of the framework is presented. Lastly, guidance on assessing model performance is offered and the importance of involving clinical stakeholders in the validation and fine-tuning process is discussed.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。