通过自改进机制提升技能学习鲁棒性，让模型在噪声数据下仍能稳定适应复杂任务。
Self-Improving Skill Learning for Robust Skill-based Meta-Reinforcement Learning
- 分高阶策略与技能优化策略，实现自引导技能精炼
- 用最大回报重标注筛选关键轨迹，聚焦有效更新
- 在长时序任务中显著优于现有方法，适合高噪声场景
元强化学习（Meta-RL）虽能快速适应新任务，但在长时程环境中面临挑战。基于技能的方法通过将状态-动作序列分解为可复用技能并采用层级决策来应对，但对离线演示数据中的噪声极为敏感，导致技能学习不稳定、性能下降。为此，我们提出自改进技能学习（SISL），通过解耦的高层策略与技能优化策略实现自引导技能精炼，并利用最大回报重标注进行技能优先级排序，使更新集中于任务相关轨迹，从而在噪声和次优数据下仍保持鲁棒稳定的适应能力。SISL有效缓解了噪声影响，实现了可靠的技能学习，在多种长时序任务上持续优于其他基于技能的元强化学习方法。代码已开源：https://epsilog.github.io/SISL。
原文摘要 · Abstract (English)
Meta-reinforcement learning (Meta-RL) facilitates rapid adaptation to unseen tasks but faces challenges in long-horizon environments. Skill-based approaches tackle this by decomposing state-action sequences into reusable skills and employing hierarchical decision-making. However, these methods are highly susceptible to noisy offline demonstrations, leading to unstable skill learning and degraded performance. To address this, we propose Self-Improving Skill Learning (SISL), which performs self-guided skill refinement using decoupled high-level and skill improvement policies, while applying skill prioritization via maximum return relabeling to focus updates on task-relevant trajectories, resulting in robust and stable adaptation even under noisy and suboptimal data. By mitigating the effect of noise, SISL achieves reliable skill learning and consistently outperforms other skill-based meta-RL methods on diverse long-horizon tasks. Our code is available at https://epsilog.github.io/SISL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。