用单一模型同时完成机器人运动策略优化与知识迁移，实现零样本部署。
Unified Locomotion Transformer with Simultaneous Sim-to-Real Transfer for Quadrupeds
- 基于Transformer的统一框架，融合强化学习与动作模仿训练
- 单阶段训练即可达成最优师生策略，无需多阶段监督
- 适合复杂变换器模型的机器人控制，显著降低仿真到现实的差距
四足机器人在复杂地形中的行走能力快速提升。深度强化学习、Transformer 及多种知识迁移技术可有效缩小仿真到现实的差距。然而，现有运动策略常采用经典师生框架，需预训练教师模型并依赖特权信息指导学生策略，导致多阶段监督，效率低下。本文提出统一运动变换器（ULT），将知识迁移与策略优化整合于单一网络中，利用强化学习、下一状态-动作预测和动作模仿，在单一训练阶段完成策略优化，实现零样本部署。实验表明，借助 ULT 可同时获得最优教师与学生策略，显著简化知识迁移流程，即使在复杂的基于 Transformer 的模型中亦表现优异。
原文摘要 · Abstract (English)
Quadrupeds have gained rapid advancement in their capability of traversing across complex terrains. The adoption of deep Reinforcement Learning (RL), transformers and various knowledge transfer techniques can greatly reduce the sim-to-real gap. However, the classical teacher-student framework commonly used in existing locomotion policies requires a pre-trained teacher and leverages the privilege information to guide the student policy. With the implementation of large-scale models in robotics controllers, especially transformers-based ones, this knowledge distillation technique starts to show its weakness in efficiency, due to the requirement of multiple supervised stages. In this paper, we propose Unified Locomotion Transformer (ULT), a new transformer-based framework to unify the processes of knowledge transfer and policy optimization in a single network while still taking advantage of privilege information. The policies are optimized with reinforcement learning, next state-action prediction, and action imitation, all in just one training stage, to achieve zero-shot deployment. Evaluation results demonstrate that with ULT, optimal teacher and student policies can be obtained at the same time, greatly easing the difficulty in knowledge transfer, even with complex transformer-based models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。