arXiv:2602.14505cs.AIcs.LG2026-02被引 3

用形式化方法验证并解释脓毒症治疗策略,发现模型依赖用药史而非病情变化。

Formally Verifying and Explaining Sepsis Treatment Policies with COOL-MC

  • 只分析策略可达状态空间,大幅降低计算开销
  • 在1.7万例患者数据上训练出最优存活率的治疗策略
  • 揭示决策依据,帮助临床调试潜在风险

安全可解释的序列决策在医疗中至关重要,但现有强化学习(RL)脓毒症治疗策略仍不透明且难验证。标准概率模型检测器因需处理全状态空间,在大型马尔可夫决策过程(MDP)中不可行,且无法解释决策原因。COOL-MC基于Storm构建,新增三项能力:仅生成由训练策略诱导的可达状态空间,形成更小的离散时间马尔可夫链(DTMC),便于验证;自动为状态添加临床有意义的原子命题;结合可解释性方法与概率计算树逻辑(PCTL)查询,揭示治疗轨迹中驱动决策的关键特征。我们在基于约1.7万例脓毒症患者记录的ICU-Sepsis MDP上验证了该方法,实现全MDP验证的严格边界约束,训练出达到最优生存概率的安全RL策略,并通过PCTL验证与可解释性分析其行为。结果表明,该策略主要依赖既往给药历史而非患者实时状况,这一缺陷在常规评估中不可见,却可通过COOL-MC暴露。研究显示,该工具可辅助临床人员在部署前审查和调试脓毒症治疗策略。

原文摘要 · Abstract (English)

Safe and interpretable sequential decision-making is critical in healthcare, yet reinforcement learning (RL) policies for sepsis treatment optimization remain opaque and difficult to verify. Standard probabilistic model checkers operate on the full state space, which becomes infeasible for larger MDPs, and cannot explain why a learned policy makes particular decisions. COOL-MC wraps the model checker Storm but adds three key capabilities: it constructs only the reachable state space induced by a trained policy, yielding a smaller discrete-time Markov chain amenable to verification even when full-MDP analysis is intractable; it automatically labels states with clinically meaningful atomic propositions; and it integrates explainability methods with probabilistic computation tree logic (PCTL) queries to reveal which features drive decisions across treatment trajectories. We demonstrate COOL-MC's capabilities on the ICU-Sepsis MDP, a benchmark derived from approximately 17,000 sepsis patient records, which serves as a case study for applying COOL-MC to the formal analysis of sepsis treatment policies. Our analysis establishes hard bounds via full MDP verification, trains a safe RL policy that achieves optimal survival probability, and analyzes its behavior via PCTL verification and explainability on the induced DTMC. This reveals, for instance, that our trained policy relies predominantly on prior dosing history rather than the patient's evolving condition, a weakness that is invisible to standard evaluation but is exposed by COOL-MC's integration of formal verification and explainability. Our results illustrate how COOL-MC could serve as a tool for clinicians to investigate and debug sepsis treatment policies before deployment.

形式化验证医疗AI可解释性强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。