arXiv:2603.02396cs.AIcs.LG2026-03

用形式化方法验证并解释强化学习在血小板库存管理中的决策逻辑。

COOL-MC: Verifying and Explaining RL Policies for Platelet Inventory Management

  • 构建可达状态的马尔可夫链,结合概率模型检测与可解释性分析
  • 政策实现2.9%缺货率和1.1%积压率,主要关注库存年龄分布
  • 揭示策略多样性及缓冲状态下单量选择机制,适合医疗供应链研究者

血小板有效期为五天,血库面临每日需求不确定的问题,需在高成本浪费与致命缺货间权衡订货决策。强化学习(RL)可学习该马尔可夫决策过程(MDP)的有效订货策略,但神经网络策略仍为黑箱,阻碍其在安全关键领域的信任与应用。本文采用COOL-MC工具,结合强化学习、概率模型检查与可解释性强化学习,对基于Haijema等人研究的血小板库存管理MDP训练出的策略进行形式化验证与解释。通过构建仅包含受训策略下可达状态的离散时间马尔可夫链(以减少内存占用),验证PCTL性质并提供特征级解释。结果表明,在200步时域内,该策略实现2.9%的缺货概率和1.1%的库存满载(潜在浪费)概率,主要关注库存年龄分布,而非星期几或待处理订单等其他特征。动作可达性分析显示策略采用多样化补货策略,多数订货量可快速达到,少数从未被选择。反事实分析表明,将中大单替换为小单后,两项安全概率几乎不变,说明这些订单仅在库存缓冲充足状态下发出。这是首个对强化学习血小板库存管理策略的形式化验证与解释,展示了COOL-MC在安全关键医疗供应链领域透明、可审计决策中的价值。

原文摘要 · Abstract (English)

Platelets expire within five days. Blood banks face uncertain daily demand and must balance ordering decisions between costly wastage from overstocking and life-threatening shortages from understocking. Reinforcement learning (RL) can learn effective ordering policies for this Markov decision process (MDP), but the resulting neural policies remain black boxes, hindering trust and adoption in safety-critical domains. We apply COOL-MC, a tool that combines RL with probabilistic model checking and explainable RL, to verify and explain a trained policy for the MDP on platelet inventory management inspired by Haijema et al. By constructing a policy-induced discrete-time Markov chain (which includes only the reachable states under the trained policy to reduce memory usage), we verify PCTL properties and provide feature-level explanations. Results show that the trained policy achieves a 2.9% stockout probability and a 1.1% inventory-full (potential wastage) probability within a 200-step horizon, primarily attends to the age distribution of inventory rather than other features such as day of week or pending orders. Action reachability analysis reveals that the policy employs a diverse replenishment strategy, with most order quantities reached quickly, while several are never selected. Counterfactual analysis shows that replacing medium-large orders with smaller ones leaves both safety probabilities nearly unchanged, indicating that these orders are placed in well-buffered inventory states. This first formal verification and explanation of an RL platelet inventory management policy demonstrates COOL-MC's value for transparent, auditable decision-making in safety-critical healthcare supply chain domains.

强化学习医疗供应链可解释性形式验证

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。