用强化学习实现机械臂快速避障轨迹规划，提升复杂环境下的实时性与稳定性。
Fast Trajectory Planner with a Reinforcement Learning-based Controller for Robotic Manipulators
- 结合视觉路径规划与强化学习，在关节空间实现避障。
- 引入动作集合与策略反馈，使PPO算法更精准稳定。
- 支持仿真到现实的迁移，适合工业场景中的动态避障任务。
在非结构化、杂乱环境中为机械臂生成无碰撞轨迹仍是重大挑战。现有方法常需额外计算求解运动学或动力学方程。本文展示无模型强化学习在关节空间无碰撞轨迹规划中优于模型方法的潜力。提出一种快速轨迹规划系统：第一部分基于视觉的路径规划，在任务空间利用大规模快速分割任意（FSA）模型与贝塞尔样条（B-spline）优化的运动学-动力学路径搜索；第二部分通过引入动作集合（AE）和策略反馈（PF），改进近端策略优化（PPO）算法，显著提升目标到达与避障的精度与稳定性。该增强使算法在多种任务中具备更强适应性，确保执行首阶段规划指令的一致性，并提升避障效率与到达准确率。实验表明，该方法在模拟到模拟（Sim-to-Sim）及模拟到现实（Sim-to-Real）迁移中均有效提升模型鲁棒性与规划效率，实现在障碍物密集环境中的实时避障与轨迹规划。项目主页：https://sites.google.com/view/ftp4rm/home
原文摘要 · Abstract (English)
Generating obstacle-free trajectories for robotic manipulators in unstructured and cluttered environments remains a significant challenge. Existing motion planning methods often require additional computational effort to generate the final trajectory by solving kinematic or dynamic equations. This paper highlights the strong potential of model-free reinforcement learning methods over model-based approaches for obstacle-free trajectory planning in joint space. We propose a fast trajectory planning system for manipulators that combines vision-based path planning in task space with reinforcement learning-based obstacle avoidance in joint space. We divide the framework into two key components. The first introduces an innovative vision-based trajectory planner in task space, leveraging the large-scale fast segment anything (FSA) model in conjunction with basis spline (B-spline)-optimized kinodynamic path searching. The second component enhances the proximal policy optimization (PPO) algorithm by integrating action ensembles (AE) and policy feedback (PF), which greatly improve precision and stability in goal-reaching and obstacle avoidance within the joint space. These PPO enhancements increase the algorithm's adaptability across diverse robotic tasks, ensuring consistent execution of commands from the first component by the manipulator, while also enhancing both obstacle avoidance efficiency and reaching accuracy. The experimental results demonstrate the effectiveness of PPO enhancements, as well as simulation-to-simulation (Sim-to-Sim) and simulation-to-reality (Sim-to-Real) transfer, in improving model robustness and planner efficiency in complex scenarios. These enhancements allow the robot to perform obstacle avoidance and real-time trajectory planning in obstructed environments. Project page available at: https://sites.google.com/view/ftp4rm/home
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。