让智能体在失败时回溯重设目标，加速学习但增加计算负担。
Preliminary Tests of the Anticipatory Classifier System with Hindsight Experience Replay
- 失败时将访问状态重设为虚拟目标，增强稀疏奖励环境下的学习信号
- 在Maze6和FrozenLake上比原模型更快掌握环境，知识获取提速显著
- 首次融合前瞻性机制与事后目标重标注，适合研究强化学习优化的读者
本文提出ACS2HER，将预判式分类器系统（ACS2）与事后经验重放（HER）机制结合。尽管ACS2通过隐式学习构建认知地图效果良好，但在稀疏奖励环境中性能易停滞。新方法在智能体未能达成主目标时触发回溯学习，将已访问状态重标记为虚拟目标，以丰富学习信号。在确定性任务Maze6和随机性任务FrozenLake上的测试表明，相较于标准ACS2，ACS2HER显著加快了知识获取与环境掌握速度。但该优势伴随计算开销上升和分类器数量大幅增长。本工作首次分析了预判机制与事后目标重标注在学习分类器系统中的结合效果。
原文摘要 · Abstract (English)
This paper introduces ACS2HER, a novel integration of the Anticipatory Classifier System (ACS2) with the Hindsight Experience Replay (HER) mechanism. While ACS2 is highly effective at building cognitive maps through latent learning, its performance often stagnates in environments characterized by sparse rewards. We propose a specific architectural variant that triggers hindsight learning when the agent fails to reach its primary goal, re-labeling visited states as virtual goals to densify the learning signal. The proposed model was evaluated on two benchmarks: the deterministic \texttt{Maze 6} and the stochastic \texttt{FrozenLake}. The results demonstrate that ACS2HER significantly accelerates knowledge acquisition and environmental mastery compared to the standard ACS2. However, this efficiency gain is accompanied by increased computational overhead and a substantial expansion in classifier numerosity. This work provides the first analysis of combining anticipatory mechanisms with retrospective goal-relabeling in Learning Classifier Systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。