用强化学习优化机械臂结构,无需公式也能自动找到最佳设计。
Task-Aware Morphology Optimization of Planar Manipulators via Reinforcement Learning
- 用强化学习通过奖励反馈优化机械臂形态,不依赖解析公式
- 在圆形路径任务中成功复现理论最优解(等长连杆+正交关节)
- 适合高维复杂路径的形态设计,比传统方法更高效可扩展
本文以吉川量度指标为基础,探索强化学习在平面机械臂形态优化中的应用。以2自由度机械臂跟踪圆弧末端轨迹为例,该问题存在已知解析最优解:两连杆等长且第二关节与第一关节正交。此设置用于验证强化学习能否仅凭奖励信号恢复最优解,而不需访问量度表达式或雅可比矩阵。比较了SAC、DDPG和PPO三种算法与网格搜索及黑箱优化器,形态由单一动作参数φ映射为连杆长度。所有方法均收敛至解析解,证明数值上可无解析结构实现最优恢复。多数形态设计问题无闭式解,随着维度上升,网格或启发式搜索成本激增。因此,强化学习被作为可扩展替代方案。将圆路径的建模扩展至椭圆和矩形路径,动作空间扩展为完整形态向量(L1, L2, theta2)。在无解析解的情况下,强化学习仍稳定收敛,而网格与黑箱方法需更大评估预算。结果表明,强化学习既能复现已知最优解,也可有效求解无解析解的形态优化问题。
原文摘要 · Abstract (English)
In this work, Yoshikawa's manipulability index is used to investigate reinforcement learning (RL) as a framework for morphology optimization in planar robotic manipulators. A 2R manipulator tracking a circular end-effector path is first examined because this case has a known analytical optimum: equal link lengths and the second joint orthogonal to the first. This serves as a validation step to test whether RL can rediscover the optimum using reward feedback alone, without access to the manipulability expression or the Jacobian. Three RL algorithms (SAC, DDPG, and PPO) are compared with grid search and black-box optimizers, with morphology represented by a single action parameter phi that maps to the link lengths. All methods converge to the analytical solution, showing that numerical recovery of the optimum is possible without supplying analytical structure. Most morphology design tasks have no closed-form solutions, and grid or heuristic search becomes expensive as dimensionality increases. RL is therefore explored as a scalable alternative. The formulation used for the circular path is extended to elliptical and rectangular paths by expanding the action space to the full morphology vector (L1, L2, theta2). In these non-analytical settings, RL continues to converge reliably, whereas grid and black-box methods require far larger evaluation budgets. These results indicate that RL is effective for both recovering known optima and solving morphology optimization problems without analytical solutions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。