让机器人身体和控制策略协同进化，提升设计效率。
Efficient Morphology-Control Co-Design via Stackelberg Proximal Policy Optimization
- 将形态与控制视为博弈关系，动态调整优化顺序。
- 在多个任务中训练更稳定，性能优于传统方法。
- 适合需要高效协同设计的机器人研发人员。
形态控制协同设计涉及对智能体身体结构与控制策略的联合优化。该问题具有双层结构：控制策略需随形态动态适应以最大化性能。现有方法通常采用单层范式，将控制策略视为固定，忽视其适应动态，导致形态更新与控制适配不同步，影响优化效率。本文从博弈论视角重新审视该问题，将形态与控制间的内在耦合建模为一种新型斯塔克尔伯格博弈。提出斯塔克尔伯格近端策略优化（Stackelberg PPO），显式引入控制适应动态至形态优化过程。通过建模这种内在耦合，本方法使形态更新与控制适应保持一致，从而稳定训练并提升学习效率。在多种协同设计任务上的实验表明，Stackelberg PPO 在训练稳定性与最终性能上均优于标准 PPO，为实现更高效的机器人设计开辟了新路径。
原文摘要 · Abstract (English)
Morphology-control co-design concerns the coupled optimization of an agent's body structure and control policy. This problem exhibits a bi-level structure, where the control dynamically adapts to the morphology to maximize performance. Existing methods typically neglect the control's adaptation dynamics by adopting a single-level formulation that treats the control policy as fixed when optimizing morphology. This can lead to inefficient optimization, as morphology updates may be misaligned with control adaptation. In this paper, we revisit the co-design problem from a game-theoretic perspective, modeling the intrinsic coupling between morphology and control as a novel variant of a Stackelberg game. We propose Stackelberg Proximal Policy Optimization (Stackelberg PPO), which explicitly incorporates the control's adaptation dynamics into morphology optimization. By modeling this intrinsic coupling, our method aligns morphology updates with control adaptation, thereby stabilizing training and improving learning efficiency. Experiments across diverse co-design tasks demonstrate that Stackelberg PPO outperforms standard PPO in both stability and final performance, opening the way for dramatically more efficient robotics designs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。