arXiv:2503.20839cs.ROcs.LG2025-03中稿 · publication at the…被引 9

用对比学习让机器人在训练中学会模仿老师,实现在真实环境中的稳定行走。

TAR: Teacher-Aligned Representations via Contrastive Learning for Quadrupedal Locomotion

  • 通过对比学习对齐师生模型的表征空间,提升感知能力。
  • 训练速度提升2倍,跨域泛化能力平均提高40%。
  • 部署时无需依赖外部信息,适合持续优化的真实机器人场景。

基于强化学习的四足行走通常采用教师-学生范式,由具备特权信息的教师指导仅依赖本体感觉的学生策略。然而,师生表征不匹配、行为克隆引发的协变量偏移以及缺乏可部署的适应能力,导致现实场景下泛化性能差。我们提出基于对比学习的教师对齐表征(TAR)框架,利用自监督对比学习将学生策略在仿真中与特权教师对齐。该方法使学生策略学习到结构化的潜在空间,在分布外(OOD)场景下表现更优,甚至超越完全特权的“教师”。实验表明,相比最先进基线,训练速度提升2倍即可达到峰值性能;在分布外场景中泛化能力平均提升40%。此外,TAR可无缝过渡至部署阶段,无需特权状态,实现了高样本效率与自适应行走,支持真实场景下的持续微调。代码与视频已开源:https://amrmousa.com/TARLoco/

原文摘要 · Abstract (English)

Quadrupedal locomotion via Reinforcement Learning (RL) is commonly addressed using the teacher-student paradigm, where a privileged teacher guides a proprioceptive student policy. However, key challenges such as representation misalignment between privileged teacher and proprioceptive-only student, covariate shift due to behavioral cloning, and lack of deployable adaptation; lead to poor generalization in real-world scenarios. We propose Teacher-Aligned Representations via Contrastive Learning (TAR), a framework that leverages privileged information with self-supervised contrastive learning to bridge this gap. By aligning representations to a privileged teacher in simulation via contrastive objectives, our student policy learns structured latent spaces and exhibits robust generalization to Out-of-Distribution (OOD) scenarios, surpassing the fully privileged "Teacher". Results showed accelerated training by 2x compared to state-of-the-art baselines to achieve peak performance. OOD scenarios showed better generalization by 40% on average compared to existing methods. Moreover, TAR transitions seamlessly into learning during deployment without requiring privileged states, setting a new benchmark in sample-efficient, adaptive locomotion and enabling continual fine-tuning in real-world scenarios. Open-source code and videos are available at https://amrmousa.com/TARLoco/.

四足行走对比学习强化学习自适应控制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。