对比四种动作空间，发现关节速度最适配视觉抓取与推动任务。
Benchmarking Action Spaces in Reinforcement Learning for Vision-based Robotic Manipulation
- 采用四类动作空间：位姿增量、位姿速度、关节位置增量、关节速度
- 关节速度在仿真到现实迁移中表现最优，运动更平滑且任务成功率更高
- 为机器人强化学习从业者提供动作空间选择实操指南
在真实世界强化学习中，动作空间的选择对运动平滑性、安全性及整体任务性能有重要影响。本文评估了位姿增量、位姿速度、关节位置增量和关节速度四种动作空间，在基于视觉的物体抓取与推动任务中的表现。策略在仿真环境中训练，并通过仿真到现实的迁移部署至真实机器人。结果表明，动作空间表示显著影响仿真到现实的性能表现。特别地，在视觉抓取与推动任务中，关节速度动作空间在运动平滑性和最终任务性能方面表现最佳。研究还为强化学习实践者在仿真与真实场景中选择动作空间提供了实用建议。
原文摘要 · Abstract (English)
In real-world reinforcement learning (RL), the choice of action space can play a key role in shaping motion smoothness, safety, and overall task performance. In this study, we evaluate pose increment, pose velocity, joint position increment, and joint velocity across two vision-based manipulation tasks: object picking and pushing. We train policies in simulation and deploy them to the real world using sim-to-real transfer. We find that action-space representation indeed significantly affects sim-to-real performance. In particular, we find that the joint velocity action space is best for the vision-based picking and pushing tasks in terms of smoothness and final task performance. We also provide practical guidance for RL practitioners in choosing action spaces for both simulation and real-world experiments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。