用强化学习控制有记忆的癌细胞群体，对抗耐药性。
Reinforcement Learning for Control of Non-Markovian Cellular Population Dynamics
- 用无模型深度强化学习设计药物剂量策略。
- 在长时记忆动态下仍能精确控制细胞群体。
- 适用于噪声环境和可变记忆强度的现实场景。
许多生物体和细胞类型（从细菌到癌细胞）具有适应波动环境的惊人能力，还能利用过往环境的记忆以更好应对曾遭遇的压力。从控制角度看，这种适应性给驱动细胞群体灭绝带来巨大挑战，具有重要临床意义。本文聚焦于表现出表型可塑性的细胞群体的药物剂量控制问题。对于状态在耐药与敏感间切换的特定动力学模型，已有精确解。但当系统参数未知且涉及复杂记忆机制时，最优解目前难以求得。为此，我们应用强化学习（RL）来识别针对新型非马尔可夫动力学的药物剂量策略。结果表明，无模型深度强化学习能够恢复精确解，并在存在长时程时间动态的情况下仍能有效控制细胞群体。为进一步验证方法在更真实场景中的表现，我们展示了在测量噪声和动态记忆强度环境下鲁棒的基于RL的控制策略。
原文摘要 · Abstract (English)
Many organisms and cell types, from bacteria to cancer cells, exhibit a remarkable ability to adapt to fluctuating environments. Additionally, cells can leverage a memory of past environments to better survive previously-encountered stressors. From a control perspective, this adaptability poses significant challenges in driving cell populations toward extinction, and thus poses an open question with great clinical significance. In this work, we focus on drug dosing in cell populations exhibiting phenotypic plasticity. For specific dynamical models switching between resistant and susceptible states, exact solutions are known. However, when the underlying system parameters are unknown, and for complex memory-based systems, obtaining the optimal solution is currently intractable. To address this challenge, we apply reinforcement learning (RL) to identify informed dosing strategies to control cell populations evolving under novel non-Markovian dynamics. We find that model-free deep RL is able to recover exact solutions and control cell populations even in the presence of long-range temporal dynamics. To further test our approach in more realistic settings, we demonstrate robust RL-based control strategies in environments with measurement noise and dynamic memory strength.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。