提出可扩展的医疗影像AI漂移监测框架,实时发现数据分布变化
Scalable Drift Monitoring in Medical Imaging AI
- 基于多模态数据一致性构建实时漂移检测机制
- 在新冠疫情期间真实数据中成功识别显著数据偏移
- 适合临床部署的低成本、高适应性监测方案
人工智能在医学影像中的应用提升了临床诊断能力,但长期可靠性面临模型漂移挑战。本文提出MMC+,在CheXstray框架基础上增强可扩展性,利用多模态数据一致性实现医疗影像AI模型的实时漂移检测。该框架通过引入基础模型MedImageInsight,无需机构特定训练即可生成高维图像嵌入,提升对多样化数据流的鲁棒性,并引入不确定性边界以更好捕捉动态临床环境中的漂移。在麻省总医院新冠疫情期间的真实数据上验证,MMC+能有效检测显著数据偏移,并与模型性能变化相关联。虽不直接预测性能下降,但可作为早期预警系统,提示AI系统偏离可接受性能范围,支持及时干预。本研究强调监测多样化数据流及结合模型性能评估的重要性,推动AI在临床环境中的可靠落地。
原文摘要 · Abstract (English)
The integration of artificial intelligence (AI) into medical imaging has advanced clinical diagnostics but poses challenges in managing model drift and ensuring long-term reliability. To address these challenges, we develop MMC+, an enhanced framework for scalable drift monitoring, building upon the CheXstray framework that introduced real-time drift detection for medical imaging AI models using multi-modal data concordance. This work extends the original framework's methodologies, providing a more scalable and adaptable solution for real-world healthcare settings and offers a reliable and cost-effective alternative to continuous performance monitoring addressing limitations of both continuous and periodic monitoring methods. MMC+ introduces critical improvements to the original framework, including more robust handling of diverse data streams, improved scalability with the integration of foundation models like MedImageInsight for high-dimensional image embeddings without site-specific training, and the introduction of uncertainty bounds to better capture drift in dynamic clinical environments. Validated with real-world data from Massachusetts General Hospital during the COVID-19 pandemic, MMC+ effectively detects significant data shifts and correlates them with model performance changes. While not directly predicting performance degradation, MMC+ serves as an early warning system, indicating when AI systems may deviate from acceptable performance bounds and enabling timely interventions. By emphasizing the importance of monitoring diverse data streams and evaluating data shifts alongside model performance, this work contributes to the broader adoption and integration of AI solutions in clinical settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。