arXiv:2602.06603cs.LG2026-02被引 1

时间重采样会严重降低医疗强化学习模型表现,最高下降60%。

The hidden risks of temporal resampling in clinical reinforcement learning

  • 用不同时间间隔重采样临床数据,影响模型决策逻辑。
  • 4小时重采样使所有模型表现低于基线,性能下降超60%。
  • 适合关注医疗RL真实部署风险的研究者阅读。

强化学习(RL)在医疗领域常采用离线强化学习(ORL),基于回顾性观测数据训练。为处理不规则临床记录,研究者常将数据按固定时间间隔重采样(即分箱)。但这种处理方式会引入虚构的临床场景,尤其在决策时间随机性强的情况下。本研究使用FDA认证的UVA/Padova模拟器,构建30名1型糖尿病虚拟患者,加入随机决策间隔生成训练数据。对比原始数据与10分钟、2小时、4小时分箱数据,训练三种ORL算法。部署后发现,时间重采样使模型性能最多下降60%,4小时分箱下所有智能体表现均劣于数据集基线。而事后评估在重采样数据上高估回报1.5至3倍。建议未来研究优先使用自然决策间隔数据,以保障模型安全落地。

原文摘要 · Abstract (English)

Reinforcement learning (RL) is a type of artificial intelligence for making optimal choices. In healthcare, researchers generally use offline RL (ORL), where models are trained and evaluated from retrospective observational data. To accommodate inherently irregular clinical records, researchers often resample the data into uniform time intervals before training (known as binning). However, discretised data presents the model with a fictional representation of clinical scenarios, especially where unpredictable decision timings are common. As these models lack robust trial evidence, we chose to explore the effects of this further by conducting an in silico clinical trial using 30 virtual patients with type 1 diabetes from the FDA-approved UVA/Padova simulator. The simulator was modified to include stochastic intervals between decisions and used to generate a training dataset for offline RL. We trained three ORL algorithms on both the unprocessed dataset and equivalent datasets resampled at 10-minute, 2-hour, and 4-hour intervals. When deployed back into the simulated environment, temporal resampling was found to reduce model performance by up to 60% relative to unprocessed data, with 4-hour binning causing all agents to perform worse than the dataset's baseline. Retrospective evaluation on resampled data actively obscured this effect, predicting 1.5-3x better returns than agents achieved in practice. We recommend that future research in this area prioritises datasets with natural clinical timings between decisions, which may be a necessary step before these models can be safely deployed into patient care.

强化学习医疗AI数据重采样糖尿病管理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。