arXiv:2605.31360cs.LGcs.AI2026-05

dashi工具可量化分析数据分布偏移,提升医疗AI的可靠性与安全性。

dashi: A Python library for Dataset Shift Characterization to Support Trustworthy AI Development and Deployment

  • 基于信息几何与非参数统计,支持无监督与有监督的偏移检测
  • 在妊娠糖尿病等3个真实医疗案例中验证性能下降可被精准定位
  • 适合医疗、金融等领域关注数据质量与模型可信度的研究者使用

人工智能生命周期需要深入理解底层数据动态,以实现稳健、安全且低成本的开发与应用。数据集偏移指训练与测试数据分布的变化,无论随时间(时序)或跨不同来源(多源),均会严重降低模型性能并影响数据质量。这在医疗AI中尤为关键,因患者安全与基本权利可能因训练和运行阶段的不可控偏移而受损。尽管协变量、先验和概念偏移的理论基础已成熟,但缺乏易用且全面的分析工具。本文提出dashi——一个开源Python库,用于探索、量化和表征数据集偏移。该库采用双路径:无监督方法利用信息几何与非参数统计流形刻画数据变异性(如信息几何时序图、全局概率偏离度与源概率离群度),有监督方法则量化模型性能退化。两者均可在用户定义的时间段或域/源批次上运行。我们在妊娠糖尿病、新冠和急诊调度三个模拟与真实医疗案例中展示了dashi的实用性。通过交互式可视化与变异度量,dashi支持全生命周期可信性评估,提升机器学习流程的鲁棒性与安全性。

原文摘要 · Abstract (English)

The Artificial Intelligence (AI) life cycle requires a thorough understanding of the underlying data dynamics for robust, safe and cost-effective AI development and use. Dataset shifts are defined as changes between train and test data distributions. Whether occurring over time (temporal) or across different sites (multi-source), they can severely degrade model performance and compromise data quality. This is particularly important in health AI, where the safety and fundamental rights of patients can be severely affected by uncontrolled shifts both at training and operational stages. While the theoretical foundations of covariate, prior, and concept shifts are well established, there is a lack of accessible and comprehensive software tools to perform their analysis. We introduce dashi, an open-source Python library designed for the exploration, quantification, and characterization of dataset shifts. dashi provides a dual approach: an unsupervised approach that leverages information geometry and non-parametric statistical manifolds to data variability characterization and analysis (e.g., Information Geometric Temporal plots and Multi-Source Variability metrics like Global Probabilistic Deviation and Source Probabilistic Outlyingness), and a supervised approach that quantifies and characterizes model performance degradation. Both unsupervised and supervised approaches work across user-defined temporal and domain/source batches. We demonstrate the utility of dashi on three simulated and real-world health AI case studies on gestational diabetes mellitus, COVID-19 and emergency medical dispatch. By providing interactive visual analytics and variability metrics, dashi supports trustworthiness of AI life cycle stages enabling robust and safe machine learning pipelines through the assessment of data coherence and AI performance.

数据偏移医疗AI可信AIPython工具

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。