用量子桥接技术生成真实治疗轨迹,提升离线强化学习在个性化医疗中的表现。
Treatment Stitching with Schrödinger Bridge for Enhancing Offline Reinforcement Learning in Adaptive Treatment Strategies
- 通过智能拼接相似患者状态的治疗片段,生成新数据。
- 在状态差异大时用薛定谔桥生成平滑过渡轨迹,提升数据多样性。
- 保持临床合理性,适合医疗决策类离线强化学习研究者。
自适应治疗策略(ATS)是根据患者症状动态调整治疗方案的序列决策过程。尽管强化学习(RL)为优化ATS提供了潜力,但其传统的在线试错机制在临床环境中不可接受,因存在伤害患者的风险。离线强化学习通过仅使用历史治疗数据来学习策略,解决了这一问题,但其性能常受限于数据稀缺性——这在临床领域尤为普遍。为此,我们提出治疗拼接(TreatStitch),一种新颖的数据增强框架,通过智能拼接现有治疗数据中的片段生成临床有效的治疗轨迹。具体而言,TreatStitch在不同轨迹中识别相似的中间患者状态,并拼接对应段落;当中间状态差异过大无法直接拼接时,利用薛定谔桥方法生成平滑且能量高效的过渡轨迹以连接不相似状态。将这些合成轨迹加入原始数据集后,离线强化学习可基于更丰富的数据学习,从而提升优化ATS的能力。多组治疗数据上的实验验证了TreatStitch的有效性。此外,我们从理论上证明了该方法通过避免分布外转移维持临床有效性。
原文摘要 · Abstract (English)
Adaptive treatment strategies (ATS) are sequential decision-making processes that enable personalized care by dynamically adjusting treatment decisions in response to evolving patient symptoms. While reinforcement learning (RL) offers a promising approach for optimizing ATS, its conventional online trial-and-error learning mechanism is not permissible in clinical settings due to risks of harm to patients. Offline RL tackles this limitation by learning policies exclusively from historical treatment data, but its performance is often constrained by data scarcity-a pervasive challenge in clinical domains. To overcome this, we propose Treatment Stitching (TreatStitch), a novel data augmentation framework that generates clinically valid treatment trajectories by intelligently stitching segments from existing treatment data. Specifically, TreatStitch identifies similar intermediate patient states across different trajectories and stitches their respective segments. Even when intermediate states are too dissimilar to stitch directly, TreatStitch leverages the Schrödinger bridge method to generate smooth and energy-efficient bridging trajectories that connect dissimilar states. By augmenting these synthetic trajectories into the original dataset, offline RL can learn from a more diverse dataset, thereby improving its ability to optimize ATS. Extensive experiments across multiple treatment datasets demonstrate the effectiveness of TreatStitch in enhancing offline RL performance. Furthermore, we provide a theoretical justification showing that TreatStitch maintains clinical validity by avoiding out-of-distribution transitions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。