用离线强化学习优化重症患者肝素个性化给药,提升安全性与精准度。
Development and Validation of Heparin Dosing Policies Using an Offline Reinforcement Learning Algorithm
- 基于离线强化学习构建约束型剂量策略,融合临床经验减少分布外误差
- 在MIMIC-III数据上验证,策略能稳定控制肝素剂量于治疗范围内
- 适合医疗决策支持系统开发、重症医学与人工智能交叉研究者参考
重症监护中合理用药对患者生存至关重要。肝素用于治疗血栓及抑制凝血,但其给药复杂且敏感,受患者特征、基础疾病和药物相互作用影响,剂量不当可能导致中风或严重出血。本研究提出一种基于强化学习的个性化肝素最优给药策略,根据个体情况可靠地将剂量维持在治疗范围内。采用批处理约束策略,在离线强化学习环境中最小化分布外误差,并有效整合现有临床决策。通过加权重要性采样(off-policy evaluation)评估策略有效性,并使用t-SNE分析状态表示与Q值的关系。基于MIMIC-III数据库进行定量与定性分析,验证了该方法的可行性。借助先进机器学习与大规模临床数据,本研究提升了肝素管理实践,为医学智能决策工具的发展树立范例。
原文摘要 · Abstract (English)
Appropriate medication dosages in the intensive care unit (ICU) are critical for patient survival. Heparin, used to treat thrombosis and inhibit blood clotting in the ICU, requires careful administration due to its complexity and sensitivity to various factors, including patient clinical characteristics, underlying medical conditions, and potential drug interactions. Incorrect dosing can lead to severe complications such as strokes or excessive bleeding. To address these challenges, this study proposes a reinforcement learning (RL)-based personalized optimal heparin dosing policy that guides dosing decisions reliably within the therapeutic range based on individual patient conditions. A batch-constrained policy was implemented to minimize out-of-distribution errors in an offline RL environment and effectively integrate RL with existing clinician policies. The policy's effectiveness was evaluated using weighted importance sampling, an off-policy evaluation method, and the relationship between state representations and Q-values was explored using t-SNE. Both quantitative and qualitative analyses were conducted using the Medical Information Mart for Intensive Care III (MIMIC-III) database, demonstrating the efficacy of the proposed RL-based medication policy. Leveraging advanced machine learning techniques and extensive clinical data, this research enhances heparin administration practices and establishes a precedent for the development of sophisticated decision-support tools in medicine.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。