用强化学习优化健康类App提醒时机,提升用户参与度。
Practical considerations when designing an online learning algorithm for an app-based mHealth intervention
- 根据用户打开概率动态决定发送提醒时间
- 减少无效通知,提高干预有效性
- 适合做移动医疗个性化干预的研究者参考
移动健康(mHealth)技术的普及为将强化学习融入传统临床试验设计提供了可能,使研究人员能在研究过程中学习个体化治疗策略。低盐生活2.0(LowSalt4Life 2, LS4L2)是一项针对高血压患者通过App干预降低钠摄入量的试验。其中一个试验组部署了强化学习算法,用于在参与者未来30分钟内可能打开应用时发送提醒通知,避免在先前数据表明效果减弱的时间段发送。该算法可减轻用户负担,更有效地促进行为改变。我们在实施过程中遇到了诸多挑战,现总结为模板以供未来类似试验参考,包括:(i)定义有意义的奖励函数;(ii)确定合理的优化时间尺度;(iii)构建可自动化的稳健统计模型;(iv)平衡模型灵活性与计算成本;(v)处理逐步收集数据中的缺失值问题。
原文摘要 · Abstract (English)
The ubiquitous nature of mobile health (mHealth) technology has expanded opportunities for the integration of reinforcement learning into traditional clinical trial designs, allowing researchers to learn individualized treatment policies during the study. LowSalt4Life 2 (LS4L2) is a recent trial aimed at reducing sodium intake among hypertensive individuals through an app-based intervention. A reinforcement learning algorithm, which was deployed in one of the trial arms, was designed to send reminder notifications to promote app engagement in contexts where the notification would be effective, i.e., when a participant is likely to open the app in the next 30-minute and not when prior data suggested reduced effectiveness. Such an algorithm can improve app-based mHealth interventions by reducing participant burden and more effectively promoting behavior change. We encountered various challenges during the implementation of the learning algorithm, which we present as a template to solving challenges in future trials that deploy reinforcement learning algorithms. We provide template solutions based on LS4L2 for solving the key challenges of (i) defining a relevant reward, (ii) determining a meaningful timescale for optimization, (iii) specifying a robust statistical model that allows for automation, (iv) balancing model flexibility with computational cost, and (v) addressing missing values in gradually collected data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。