用多智能体强化学习优化手术室实时调度，平衡效率与不确定性。
Multi-Agent Reinforcement Learning for Intraday Operating Rooms Scheduling under Uncertainty
- 每个手术室作为智能体，通过集中训练分散执行，联合生成无冲突排程。
- 在六间手术室、八类手术的模拟中，优于六种规则启发式策略，接近最优解。
- 可解释性强，能识别优先处理急症、合并相似手术、推迟低价值手术等策略。
术中手术排程是在不确定性下平衡择期手术量、紧急需求、延误、依赖顺序的准备时间及加班的多目标决策问题。我们将其建模为合作马尔可夫博弈，提出一种多智能体强化学习（MARL）框架，其中每个手术室（OR）为一个智能体，采用集中训练、分散执行方式，共享由近端策略优化（PPO）训练的策略，将复杂系统状态映射为动作；通过跨周期的顺序分配协议构建各手术室间的无冲突联合排程。基于混合整数预排程提供择期手术起始时间参考，施加针对各类手术的二次延迟惩罚和终端加班惩罚，形成单一奖励函数，涵盖手术量、及时性与人员负荷。在反映真实医院配置的仿真中（六间手术室，八类手术，随机紧急/急诊到达），所学策略在七个指标和三个评估子集上均优于六种规则启发式方法，并相对于事后求解的混合整数规划（MIP）最优解，量化了次优差距。策略分析揭示可解释行为：优先处理急症、批量相似手术以减少准备时间、推迟低价值择期手术。此外，在简化假设下推导了顺序分解的次优性界。讨论了局限性——如手术室同质性与未显式建模人力约束——并提出扩展方向。总体而言，该方法为实时手术室调度提供了实用、可解释且可调的、数据驱动的优化补充。
原文摘要 · Abstract (English)
Intraday surgical scheduling is a multi-objective decision problem under uncertainty-balancing elective throughput, urgent and emergency demand, delays, sequence-dependent setups, and overtime. We formulate the problem as a cooperative Markov game and propose a multi-agent reinforcement learning (MARL) framework in which each operating room (OR) is an agent trained with centralized training and decentralized execution. All agents share a policy trained via Proximal Policy Optimization (PPO), which maps rich system states to actions, while a within-epoch sequential assignment protocol constructs conflict-free joint schedules across ORs. A mixed-integer pre-schedule provides reference starting times for electives; we impose type-specific quadratic delay penalties relative to these references and a terminal overtime penalty, yielding a single reward that captures throughput, timeliness, and staff workload. In simulations reflecting a realistic hospital mix (six ORs, eight surgery types, random urgent and emergency arrivals), the learned policy outperforms six rule-based heuristics across seven metrics and three evaluation subsets, and, relative to an ex post MIP oracle, quantifies optimality gaps. Policy analytics reveal interpretable behavior-prioritizing emergencies, batching similar cases to reduce setups, and deferring lower-value electives. We also derive a suboptimality bound for the sequential decomposition under simplifying assumptions. We discuss limitations-including OR homogeneity and the omission of explicit staffing constraints-and outline extensions. Overall, the approach offers a practical, interpretable, and tunable data-driven complement to optimization for real-time OR scheduling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。