分层策略让机器人更聪明地推物，效率更高。
Push Smarter, Not Harder: Hierarchical RL-Diffusion Policy for Efficient Nonprehensile Manipulation
- 高层用强化学习选目标，低层用扩散模型生成路径
- 在复杂环境成功率超基准，路径更短更省力
- 适合做高效非抓取操作的机器人研究者
非抓取式操作（如在杂乱环境中推动物体）因接触动力学复杂且需长时规划而极具挑战。本文提出HeRD，一种分层强化学习-扩散策略，将任务分解为高层目标选择与低层轨迹生成两阶段。高层采用强化学习代理选择中间空间目标，低层则使用条件扩散模型生成可达且高效的运动轨迹。该架构融合了强化学习的长期奖励优化能力与扩散模型的生成优势。我们在2D仿真环境中评估，结果表明该方法在成功率、路径效率及多环境配置泛化性上均优于当前最优基线。实验验证了分层控制结合生成式低层规划在可扩展、目标导向的非抓取操作中的潜力。代码、文档及训练模型已开源：https://github.com/carosteven/HeRD。
原文摘要 · Abstract (English)
Nonprehensile manipulation, such as pushing objects across cluttered environments, presents a challenging control problem due to complex contact dynamics and long-horizon planning requirements. In this work, we propose HeRD, a hierarchical reinforcement learning-diffusion policy that decomposes pushing tasks into two levels: high-level goal selection and low-level trajectory generation. We employ a high-level reinforcement learning (RL) agent to select intermediate spatial goals, and a low-level goal-conditioned diffusion model to generate feasible, efficient trajectories to reach them. This architecture combines the long-term reward maximizing behaviour of RL with the generative capabilities of diffusion models. We evaluate our method in a 2D simulation environment and show that it outperforms the state-of-the-art baseline in success rate, path efficiency, and generalization across multiple environment configurations. Our results suggest that hierarchical control with generative low-level planning is a promising direction for scalable, goal-directed nonprehensile manipulation. Code, documentation, and trained models are available: https://github.com/carosteven/HeRD.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。