用稀疏非线性动力学识别法，快速构建高精度可解释的强化学习仿真环境。
Learning from Less: SINDy Surrogates in RL
- 基于SINDy算法从少量交互中提取环境动态规律。
- 仅需75次交互即可达0.997以上状态相关性，误差低至3.11e-6。
- 适合资源有限场景下高效训练强化学习智能体。
本文提出一种基于稀疏非线性动力学识别（SINDy）算法构建强化学习代理环境的方法。在OpenAI Gym的Mountain Car与Lunar Lander环境中，仅通过75次交互（Mountain Car）和1000次交互（Lunar Lander），SINDy代理模型即可实现超过0.997的状态相关性，速度误差低至3.11e-6，位置误差低至1.42e-6。相比原环境，训练智能体所需总步数减少20%-35%（Mountain Car：65,075 vs. 100,000；Lunar Lander：801,000 vs. 1,000,000），性能相当且收敛模式一致。该方法为模型驱动型强化学习提供了高效、准确、可解释的替代方案。
原文摘要 · Abstract (English)
This paper introduces an approach for developing surrogate environments in reinforcement learning (RL) using the Sparse Identification of Nonlinear Dynamics (SINDy) algorithm. We demonstrate the effectiveness of our approach through extensive experiments in OpenAI Gym environments, particularly Mountain Car and Lunar Lander. Our results show that SINDy-based surrogate models can accurately capture the underlying dynamics of these environments while reducing computational costs by 20-35%. With only 75 interactions for Mountain Car and 1000 for Lunar Lander, we achieve state-wise correlations exceeding 0.997, with mean squared errors as low as 3.11e-06 for Mountain Car velocity and 1.42e-06 for LunarLander position. RL agents trained in these surrogate environments require fewer total steps (65,075 vs. 100,000 for Mountain Car and 801,000 vs. 1,000,000 for Lunar Lander) while achieving comparable performance to those trained in the original environments, exhibiting similar convergence patterns and final performance metrics. This work contributes to the field of model-based RL by providing an efficient method for generating accurate, interpretable surrogate environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。