用强化学习平台研究疫情中行为奖励设计，发现引导和遵守激励最关键
Reward Engineering for Spatial Epidemic Simulations: A Reinforcement Learning Platform for Individual Behavioral Learning
- 构建可配置的疫情仿真环境，支持多种奖励函数对比测试
- 潜在场奖励使智能体存活率最高，学会规避风险并遵守防疫措施
- 适合研究疫情行为建模、奖励设计与智能体学习的科研人员
我们提出ContagionRL，一个兼容Gymnasium的强化学习平台，专为空间疫情模拟中的奖励工程设计。与依赖固定规则的传统基于代理模型不同，该平台能系统评估奖励函数设计对智能体生存策略的影响。ContagionRL整合了可配置的空间SIRS+D流行病模型，支持在有限可观测性、不同移动模式及异质人口动态等条件下测试奖励函数。我们在多种强化学习算法（PPO、SAC、A2C）上评估五种不同奖励设计，包括稀疏生存奖励到新型势场方法。系统消融实验表明，方向引导和明确遵守激励是鲁棒策略学习的关键。在不同感染率、网格大小、可见性约束和移动模式下，奖励选择显著影响智能体行为与生存结果。采用势场奖励的智能体表现最优，持续实现最高防疫依从性，并发展出复杂的空间规避策略。平台模块化设计支持系统探索奖励-行为关系，填补了此类模型中奖励工程研究不足的空白。代码已开源。
原文摘要 · Abstract (English)
We present ContagionRL, a Gymnasium-compatible reinforcement learning platform specifically designed for systematic reward engineering in spatial epidemic simulations. Unlike traditional agent-based models that rely on fixed behavioral rules, our platform enables rigorous evaluation of how reward function design affects learned survival strategies across diverse epidemic scenarios. ContagionRL integrates a spatial SIRS+D epidemiological model with configurable environmental parameters, allowing researchers to stress-test reward functions under varying conditions including limited observability, different movement patterns, and heterogeneous population dynamics. We evaluate five distinct reward designs, ranging from sparse survival bonuses to a novel potential field approach, across multiple RL algorithms (PPO, SAC, A2C). Through systematic ablation studies, we identify that directional guidance and explicit adherence incentives are critical components for robust policy learning. Our comprehensive evaluation across varying infection rates, grid sizes, visibility constraints, and movement patterns reveals that reward function choice dramatically impacts agent behavior and survival outcomes. Agents trained with our potential field reward consistently achieve superior performance, learning maximal adherence to non-pharmaceutical interventions while developing sophisticated spatial avoidance strategies. The platform's modular design enables systematic exploration of reward-behavior relationships, addressing a knowledge gap in models of this type where reward engineering has received limited attention. ContagionRL is an effective platform for studying adaptive behavioral responses in epidemic contexts and highlight the importance of reward design, information structure, and environmental predictability in learning. Our code is publicly available at https://github.com/redradman/ContagionRL
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。