用强化学习优化术后脓毒症肝素治疗，降低死亡率至0.74%。
TECM*: A Data-Driven Assessment to Reinforcement Learning Methods and Application to Heparin Treatment Strategy for Surgical Sepsis
- 将SOFA评分转为连续的cxSOFA，提升状态与奖励函数精度。
- 提出TECM评估矩阵，发现CQL算法使死亡率从1.83%降至0.74%。
- 适合临床决策支持系统研发者及重症医学研究者参考。
脓毒症是由严重感染引起的危及生命的疾病，导致急性器官功能障碍。本研究提出一种数据驱动的评估指标和连续奖励函数，以优化外科脓毒症患者的个性化肝素治疗。基于MIMIC-IV v1.0和eICU v2.0数据库的数据，训练队列包含接受未分馏肝素（UFH）治疗的腹部手术后脓毒症患者。我们构建了一种新的强化学习框架：将离散的SOFA评分转化为连续的cxSOFA，实现更精细的状态与奖励设计；通过逐步分析定义“优”或“劣”的治疗策略；提出治疗效果对比矩阵（TECM），类比分类任务中的混淆矩阵，用于评估治疗方案。应用Q-Learning、DQN、DDQN、BCQ和CQL等不同强化学习算法进行优化，并进行全面评估。结果显示，基于cxSOFA-CQL的模型表现最佳，死亡率由1.83%降至0.74%，平均住院天数从11.11天缩短至9.42天。TECM在各模型间保持一致结果，体现框架稳健性。结论表明，该强化学习框架可实现可解释且鲁棒的肝素治疗优化，连续的cxSOFA评分与TECM评估方法为治疗方案提供更细致的判断依据，有望改善临床结局与决策支持可靠性。
原文摘要 · Abstract (English)
Objective: Sepsis is a life-threatening condition caused by severe infection leading to acute organ dysfunction. This study proposes a data-driven metric and a continuous reward function to optimize personalized heparin therapy in surgical sepsis patients. Methods: Data from the MIMIC-IV v1.0 and eICU v2.0 databases were used for model development and evaluation. The training cohort consisted of abdominal surgery patients receiving unfractionated heparin (UFH) after postoperative sepsis onset. We introduce a new RL-based framework: converting the discrete SOFA score to a continuous cxSOFA for more nuanced state and reward functions; Second, defining "good" or "bad" strategies based on cxSOFA by a stepwise manner; Third, proposing a Treatment Effect Comparison Matrix (TECM), analogous to a confusion matrix for classification tasks, to evaluate the treatment strategies. We applied different RL algorithms, Q-Learning, DQN, DDQN, BCQ and CQL to optimize the treatment and comprehensively evaluated the framework. Results: Among the AI-derived strategies, the cxSOFA-CQL model achieved the best performance, reducing mortality from 1.83% to 0.74% with the average hospital stay from 11.11 to 9.42 days. TECM demonstrated consistent outcomes across models, highlighting robustness. Conclusion: The proposed RL framework enables interpretable and robust optimization of heparin therapy in surgical sepsis. Continuous cxSOFA scoring and TECM-based evaluation provide nuanced treatment assessment, showing promise for improving clinical outcomes and decision-support reliability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。