让智能体重复成功动作序列，快速提升学习效率。
Repetition as Reinforcement: Enhancing Sample Efficiency via Instant Episode Repetition in Reinforcement Learning
- 成功经验立即重复，强化行为记忆
- 在多个环境上提升学习速度与性能
- 适合追求高效训练的强化学习研究者
重复是人类学习的基础机制，反复回顾成功经验能加强记忆、巩固技能并提升未来表现。受此启发，我们提出即时回合重复（IER），一种简单新颖的机制：在环境交互过程中，一旦识别出高奖励回合，就立即重复其动作序列若干次，通过与环境的再次互动强化有价值行为。与传统的经验回放和自模仿学习不同，IER直接作用于数据收集过程。我们将IER集成到SAC和TD3等先进算法中，在连续控制基准任务（包括MuJoCo、DeepMind Control Suite）及机器人动态物体搬运真实任务上进行评估。实验结果表明，该机制显著优于标准方法和基于自模仿的学习基线，有效提升了样本效率。
原文摘要 · Abstract (English)
Repetition is a fundamental mechanism in human learning, where revisiting successful experiences strengthens memory, consolidates skills, and improves future performance. Motivated by this biological principle, we introduce Instant Episode Repetition (IER), a simple and novel mechanism that improves sample efficiency by immediately repeating action sequences from successful episodes during environment interaction. Unlike conventional approaches such as Experience Replay and Self-Imitation Learning (SIL), which passively reuse past experience during training updates, IER directly influences the data collection process. Upon identifying a high-reward episode, the agent repeats its action sequence for a fixed number of subsequent episodes, reinforcing valuable behaviors through renewed interaction with the environment. We integrate IER into state-of-the-art SAC and TD3 algorithms and evaluate its effectiveness on continuous-control benchmarks, including MuJoCo, the DeepMind Control Suite, and a real-world dynamic object translation task with a robotic manipulator. Experimental results demonstrate that this simple mechanism improves learning performance over standard and self-imitation-based baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。