用低秩适配安全高效微调机器人行走策略，实测提速46.5%且零安全违规。
SLowRL: Safe Low-Rank Adaptation Reinforcement Learning for Locomotion
- 结合低秩适配与恢复策略，实现硬件上的安全微调。
- 微调时间减少46.5%，真实场景中几乎无安全违规。
- 仅需秩1适配即可恢复预训练性能，适合动态机器人应用。
仿真到现实的运动策略迁移常因仿真-现实差距导致性能下降。直接在硬件上粗略微调存在机械损坏风险且样本效率低。本文提出SLowRL框架，通过低秩适配（LoRA）结合训练时的安全恢复策略，实现动态行走任务在真实硬件上的安全高效微调。在仿真和真实Unitree Go2四足机器人上评估跳行与慢跑任务，结果表明该方法相比标准近端策略优化（PPO）基线，微调时间减少46.5%，且安全违规接近零。值得注意的是，仅使用秩1适配即可在真实世界中恢复预训练性能，同时保持稳定安全的微调过程。这些结果证明了该方法在动态机器人应用中的实用性。
原文摘要 · Abstract (English)
Sim-to-real transfer of locomotion policies often leads to performance degradation due to the inevitable sim-to-real gap. Naively fine-tuning these policies directly on hardware is problematic, as it poses risks of mechanical failure and suffers from high sample inefficiency. In this paper, we address the challenge of safely and efficiently fine-tuning reinforcement learning (RL) policies for dynamic locomotion tasks. Specifically, we focus on fine-tuning policies learned in simulation directly on hardware, while explicitly enforcing safety constraints. In doing so, we introduce SLowRL, a framework that combines Low-Rank Adaptation (LoRA) with training-time safety enforcement via a recovery policy. We evaluate our method both in simulation and on a real Unitree Go2 quadruped robot for jump and trot tasks. Experimental results show that our method achieves a $46.5\%$ reduction in fine-tuning time and near-zero safety violations compared to standard proximal policy optimization (PPO) baselines. Notably, we find that a rank-1 adaptation alone is sufficient to recover pre-trained performance in the real world, while maintaining stable and safe real-world fine-tuning. These results demonstrate the practicality of safe, efficient fine-tuning for dynamic real-world robotic applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。