用可穿戴设备数据,让老年人更安全地动起来。
Diffusion Policies with Offline and Inverse Reinforcement Learning for Promoting Physical Activity in Older Adults Using Wearable Sensors
- 用柯尔莫哥洛夫-阿诺德网络估计奖励函数
- 扩散策略在离线强化学习中提升动作优化效率
- 适合高跌倒风险老人的个性化运动干预
利用真实临床数据进行离线强化学习(RL)在医疗AI中日益受到关注,但存在诸多挑战:直接定义奖励困难,逆强化学习(IRL)难以从复杂环境中准确推断奖励函数,且离线RL常与真实人类行为不一致。为解决高跌倒风险老年人通过可穿戴传感器促进身体活动的离线强化学习难题,本文提出基于柯尔莫哥洛夫-阿诺德网络与扩散策略的离线逆强化学习方法(KANDI)。该方法通过柯尔莫哥洛夫-阿诺德网络从低跌倒风险老年人(专家)的自由生活行为中学习奖励函数,结合基于扩散的策略在演员-评论家框架下实现动作生成与精细化调整,提升离线强化学习效率。我们在来自物理反馈运动计划(PEER)研究的两臂临床试验数据上评估了KANDI,验证其在跌倒风险干预中的实际应用价值。此外,KANDI在D4RL基准测试中优于现有先进方法。结果表明,KANDI能有效应对医疗场景下离线强化学习的关键挑战,为健康干预提供可行方案。
原文摘要 · Abstract (English)
Utilizing offline reinforcement learning (RL) with real-world clinical data is getting increasing attention in AI for healthcare. However, implementation poses significant challenges. Defining direct rewards is difficult, and inverse RL (IRL) struggles to infer accurate reward functions from expert behavior in complex environments. Offline RL also encounters challenges in aligning learned policies with observed human behavior in healthcare applications. To address challenges in applying offline RL to physical activity promotion for older adults at high risk of falls, based on wearable sensor activity monitoring, we introduce Kolmogorov-Arnold Networks and Diffusion Policies for Offline Inverse Reinforcement Learning (KANDI). By leveraging the flexible function approximation in Kolmogorov-Arnold Networks, we estimate reward functions by learning free-living environment behavior from low-fall-risk older adults (experts), while diffusion-based policies within an Actor-Critic framework provide a generative approach for action refinement and efficiency in offline RL. We evaluate KANDI using wearable activity monitoring data in a two-arm clinical trial from our Physio-feedback Exercise Program (PEER) study, emphasizing its practical application in a fall-risk intervention program to promote physical activity among older adults. Additionally, KANDI outperforms state-of-the-art methods on the D4RL benchmark. These results underscore KANDI's potential to address key challenges in offline RL for healthcare applications, offering an effective solution for activity promotion intervention strategies in healthcare.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。