用符号执行自动划分状态空间，提升强化学习效率与可靠性。
Symbolic State Partitioning for Reinforcement Learning
- 通过符号执行提取环境动态，生成结构化状态划分。
- 在稀疏奖励任务中显著提升策略学习效果与状态覆盖度。
- 适合需要高效探索的复杂连续状态强化学习场景。
表格型强化学习方法无法直接处理连续状态空间。一种解决方案是状态空间划分。良好的划分能促进学习过程中的泛化并更高效地利用先验经验，从而加快学习速度并生成更可靠的策略。然而，划分会引入近似误差，尤其在状态分量间存在非线性关系时危害更大。理想的划分应尽可能粗粒度，同时保留问题关键的结构特征。本文通过符号执行从环境动态中提取状态划分。实验表明，符号化划分提升了状态空间对环境行为的覆盖度，并使强化学习在稀疏奖励设置下表现更优。评估涵盖划分精度、可扩展性、智能体学习性能及所学策略的状态覆盖度。
原文摘要 · Abstract (English)
Tabular reinforcement learning methods cannot operate directly on continuous state spaces. One solution for this problem is to partition the state space. A good partitioning enables generalization during learning and more efficient exploitation of prior experiences. Consequently, the learning process becomes faster and produces more reliable policies. However, partitioning introduces approximation, which is particularly harmful in the presence of nonlinear relations between state components. An ideal partition should be as coarse as possible, while capturing the key structure of the state space for the given problem. This work extracts partitions from the environment dynamics by symbolic execution. We show that symbolic partitioning improves state space coverage with respect to environmental behavior and allows reinforcement learning to perform better for sparse rewards. We evaluate symbolic state space partitioning with respect to precision, scalability, learning agent performance and state space coverage for the learnt policies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。