提出多分辨率技能框架，让智能体更灵活地选择远近目标
MRS: Multi-Resolution Skills for HRL Agents
- 设计多个不同时间尺度的目标预测模块，按需选择
- 在多个任务上显著缩小了分层强化学习与普通方法的性能差距
- 适合需要精细控制与长程规划的复杂机器人任务
分层强化学习将策略分解为管理器和执行者，虽支持长时序规划，但对敏捷性任务表现不佳。我们发现根本原因在于：子目标表示通常未考虑可达性或与当前状态的时间距离，导致局部子目标选择不精准。进一步表明，最优子目标距离取决于任务与状态：近目标利于精确控制但放大预测噪声，远目标则使动作更平滑但牺牲几何精度。为此提出多分辨率技能（MRS），学习多个固定时序范围的目标预测模块，并通过联合训练的元控制器根据当前状态动态选择。MRS在DeepMind Control Suite、Gym-Robotics及长周期AntMaze任务中持续优于固定分辨率基线，显著缩小了分层与非分层方法间的性能差距。
原文摘要 · Abstract (English)
Hierarchical reinforcement learning (HRL) decomposes the policy into a manager and a worker, enabling long-horizon planning but introducing a performance gap on tasks requiring agility. We identify a root cause: in subgoal-based HRL, the manager's goal representation is typically learned without constraints on reachability or temporal distance from the current state, preventing precise local subgoal selection. We further show that the optimal subgoal distance is both task- and state-dependent: nearby subgoals enable precise control but amplify prediction noise, while distant subgoals produce smoother motion at the cost of geometric precision. We propose Multi-Resolution Skills (MRS), which learns multiple goal-prediction modules each specialized to a fixed temporal horizon, with a jointly trained meta-controller that selects among them based on the current state. MRS consistently outperforms fixed-resolution baselines and significantly reduces the performance gap between HRL and non-HRL state-of-the-art on DeepMind Control Suite, Gym-Robotics, and long-horizon AntMaze tasks. [Project page: https://sites.google.com/view/multi-res-skills/home]
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。