用随机固定网络实现少参数四足机器人控制,效果媲美传统方法。
RANDPOL: Parameter-Efficient End-to-End Quadruped Locomotion via Randomized Policy Learning
- 隐藏层随机初始化并固定,仅训练最后线性层,大幅减少可调参数。
- 在Go2机器人上达到与PPO相当的行走性能,参数量减少90%以上。
- 适合资源受限场景,如嵌入式部署或快速训练,且支持零样本迁移。
现代基于学习的四足机器人运动控制器通常依赖于参数量庞大的全可训练深度神经网络。本文探索一种不同的端到端控制设计:能否在极小的可训练参数空间下实现有效的四足运动?我们提出随机策略学习(RANDPOL),其动作和评价网络的隐藏层随机初始化后固定,仅训练最终线性读出层。该方法通过固定随机基函数保持非线性表达能力,同时显著降低优化维度。RANDPOL建立在随机函数逼近的数学基础上,为使用固定随机非线性特征作为表达函数类提供了理论支撑。我们在Unitree Go2机器人上评估了端到端运动控制性能,并与近端策略优化(PPO)进行对比。结果表明,RANDPOL以极少的可训练参数实现了接近PPO的运动表现,每轮迭代计算时间更短,且具备优异的性能-复杂度权衡。进一步实验验证了所学策略在物理机器人上的零样本模拟到现实迁移成功,可在用户输入前向速度和偏航率指令下稳定运行。这些结果表明,对于结构化机器人控制任务,降低可训练复杂度仍可兼容高效仿真与真实世界表现。
原文摘要 · Abstract (English)
Modern learning-based locomotion controllers typically rely on fully trainable deep neural networks with a large number of parameters. This paper studies a different design point for end-to-end control: whether effective quadruped locomotion can be achieved with a drastically reduced trainable parameter space. We present RANDomized POlicy Learning (RANDPOL), a policy learning approach in which the hidden layers of the actor and critic are randomly initialized and fixed, while only the final linear readout is trained. This yields a parameter-efficient controller class that retains nonlinear expressiveness through a fixed random basis while substantially reducing the dimension of the optimization problem. RANDPOL is supported by the mathematical foundation of randomized function approximation, which provides a principled basis for using fixed random nonlinear features as expressive function classes. We evaluate RANDPOL on end-to-end locomotion control for the Unitree Go2 quadruped and compare it with Proximal Policy Optimization (PPO). The results show that RANDPOL attains comparative locomotion performance with far fewer trainable parameters, lower learning-phase computation time per iteration, and a favorable performance-complexity trade-off. We further demonstrate successful zero-shot sim-to-real transfer of the learned RANDPOL controller on the physical Unitree Go2 under user-issued forward-velocity and yaw-rate commands. These results indicate that, for structured robotic control problems, reducing trainable complexity can remain compatible with effective simulated and real-world performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。