提出在线表格强化学习的多重插补方法，解决实时数据缺失问题。
Missing Data Multiple Imputation for Tabular Q-Learning in Online RL
- 构建多个并行插补路径，动态更新以适应在线学习需求。
- 在网格世界实验中，多重插补显著提升学习效率与策略稳定性。
- 适合处理高频率缺失数据的在线强化学习场景，如实时决策系统。
在线强化学习中的缺失数据问题比传统表格数据或离线策略学习更具挑战性。由于需在每个时间步同时完成插补和决策，插补不能延迟至数据足够多时再进行，且后续数据收集与学习依赖于先前的插补结果。本文提出完全在线的插补集成方法，发现维持多个插补路径有助于在捕捉缺失不确定性与在线环境下的效率之间取得平衡。研究了多种将这些路径融入学习与动作选择的方法，并在包含多种缺失模式的网格世界实验中，提供了初步证据：多重插补路径可构成一种简单高效的在线缺失数据强化学习框架。
原文摘要 · Abstract (English)
Missing data in online reinforcement learning (RL) poses challenges compared to missing data in standard tabular data or in offline policy learning. The need to impute and act at each time step means that imputation cannot be put off until enough data exist to produce stable imputation models. It also means future data collection and learning depend on previous imputations. This paper proposes fully online imputation ensembles. We find that maintaining multiple imputation pathways may help balance the need to capture uncertainty under missingness and the need for efficiency in online settings. We consider multiple approaches for incorporating these pathways into learning and action selection. Using a Grid World experiment with various types of missingness, we provide preliminary evidence that multiple imputation pathways may be a useful framework for constructing simple and efficient online missing data RL methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。