用潜空间想象病人状态,提升医疗决策的个性化与效果。
medDreamer: Model-Based Reinforcement Learning with Latent Imagination on Complex EHRs for Clinical Decision Support
- 基于潜空间构建患者状态模型,处理不规则采样和缺失数据。
- 在脓毒症与呼吸机治疗任务中,临床结果与离线评估均优于基线。
- 适合需要个性化、动态调整的临床决策场景,如重症监护。
及时且个性化的治疗决策在多种医疗场景中至关重要,因患者反应差异大且随时间演变。用于支持决策的临床数据常为不规则采样,缺失频率本身可能隐含病情信息。现有基于强化学习的临床决策系统通常忽略缺失模式,通过粗粒度离散化和简单插补扭曲数据,且多为无模型方法,严重依赖回顾性数据,导致探索不足与历史行为偏差。为此,我们提出medDreamer,一种新型基于模型的强化学习框架,用于个性化治疗推荐。medDreamer包含一个世界模型,配备自适应特征融合模块,可从不规则数据中模拟潜空间患者状态,并采用两阶段策略,在真实与想象轨迹的混合数据上训练。该方法可学习超越历史决策次优性的最优策略,同时保持与真实临床数据的一致性。我们在两个大规模电子健康记录(EHR)数据集上,针对脓毒症和机械通气治疗任务进行了评估。全面实验表明,medDreamer在临床结果和离线评估指标上均显著优于模型无关及基于模型的基线方法。
原文摘要 · Abstract (English)
Timely and personalized treatment decisions are essential across a wide range of healthcare settings where patient responses can vary significantly and evolve over time. Clinical data used to support these treatment decisions are often irregularly sampled, where missing data frequencies may implicitly convey information about the patient's condition. Existing Reinforcement Learning (RL) based clinical decision support systems often ignore the missing patterns and distort them with coarse discretization and simple imputation. They are also predominantly model-free and largely depend on retrospective data, which could lead to insufficient exploration and bias by historical behaviors. To address these limitations, we propose medDreamer, a novel model-based reinforcement learning framework for personalized treatment recommendation. medDreamer contains a world model with an Adaptive Feature Integration module that simulates latent patient states from irregular data and a two-phase policy trained on a hybrid of real and imagined trajectories. This enables learning optimal policies that go beyond the sub-optimality of historical clinical decisions, while remaining close to real clinical data. We evaluate medDreamer on both sepsis and mechanical ventilation treatment tasks using two large-scale Electronic Health Records (EHRs) datasets. Comprehensive evaluations show that medDreamer significantly outperforms model-free and model-based baselines in both clinical outcomes and off-policy metrics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。