用可解释的不确定性评估优化呼吸机设置,提升重症患者生存率。
Distribution-Free Uncertainty Quantification in Mechanical Ventilation Treatment: A Conformal Deep Q-Learning Framework
- 结合置信预测与深度强化学习,实现无需分布假设的不确定性量化。
- 在MIMIC-IV数据上使90天存活率提升,优于医生和基线模型。
- 输出决策置信度,适合临床部署与人机协作场景。
机械通气(MV)是重症监护病房(ICU)中关键的生命支持手段。然而,由于需平衡个体生理需求与不良结局风险,最佳通气参数难以确定。本文提出ConformalDQN,一种基于置信预测的无分布假设深度强化学习方法,用于优化ICU中的机械通气。通过将置信预测与深度强化学习结合,该方法缓解了离线场景下Q值过估计及分布外动作的问题。模型在MIMIC-IV数据库的患者记录上训练与评估,扩展了Double DQN架构并引入置信预测器,采用复合损失函数平衡Q学习与校准的概率估计。该设计使模型能感知不确定性的行动选择,在陌生状态避免潜在有害操作,并在分布偏移时更保守应对。相比医生策略、策略约束方法和行为克隆等基线模型,ConformalDQN始终推荐处于临床安全有效范围内的参数,显著提高90天生存率。此外,该方法提供可解释的决策置信度,对临床采纳与人机协同应用至关重要。
原文摘要 · Abstract (English)
Mechanical Ventilation (MV) is a critical life-support intervention in intensive care units (ICUs). However, optimal ventilator settings are challenging to determine because of the complexity of balancing patient-specific physiological needs with the risks of adverse outcomes that impact morbidity, mortality, and healthcare costs. This study introduces ConformalDQN, a novel distribution-free conformal deep Q-learning approach for optimizing mechanical ventilation in intensive care units. By integrating conformal prediction with deep reinforcement learning, our method provides reliable uncertainty quantification, addressing the challenges of Q-value overestimation and out-of-distribution actions in offline settings. We trained and evaluated our model using ICU patient records from the MIMIC-IV database. ConformalDQN extends the Double DQN architecture with a conformal predictor and employs a composite loss function that balances Q-learning with well-calibrated probability estimation. This enables uncertainty-aware action selection, allowing the model to avoid potentially harmful actions in unfamiliar states and handle distribution shifts by being more conservative in out-of-distribution scenarios. Evaluation against baseline models, including physician policies, policy constraint methods, and behavior cloning, demonstrates that ConformalDQN consistently makes recommendations within clinically safe and relevant ranges, outperforming other methods by increasing the 90-day survival rate. Notably, our approach provides an interpretable measure of confidence in its decisions, which is crucial for clinical adoption and potential human-in-the-loop implementations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。