对比多种旋转表示在强化学习中的表现，给出可靠选择建议。
A Primer on SO(3) Action Representations in Deep Reinforcement Learning
- 用切向量表示旋转动作，提升算法稳定性。
- 不同表示影响探索效率与训练收敛性，切向量最优。
- 适用于机器人控制等需精确方向操作的任务。
许多机器人控制任务需要对方向进行操作，但SO(3)的几何结构使得这一问题复杂。由于SO(3)不存在全局光滑且最小的参数化方式，常见的欧拉角、四元数、旋转矩阵和李代数坐标均引入特定约束与失败模式。尽管这些权衡在监督学习中已有研究，但在强化学习中作为动作表示的影响仍不明确。本文系统评估了三种标准连续控制算法（PPO、SAC、TD3）在密集与稀疏奖励下的表现，分析了不同表示对探索、熵正则化及训练稳定性的影 响，并研究了从欧氏网络输出获得有效旋转的不同投影机制。在一系列机器人基准测试中，量化了表示选择的实际影响，提炼出可直接实施的旋转动作选择指南。结果表明，表示所诱导的几何结构显著影响探索与优化过程，以局部坐标系下的切向量表示动作效果最稳定可靠。项目网页与代码已公开。
原文摘要 · Abstract (English)
Many robotic control tasks require policies to act on orientations, yet the geometry of SO(3) makes this nontrivial. Because SO(3) admits no global, smooth, minimal parameterization, common representations such as Euler angles, quaternions, rotation matrices, and Lie algebra coordinates introduce distinct constraints and failure modes. While these trade-offs are well studied for supervised learning, their implications for actions in reinforcement learning remain unclear. We systematically evaluate SO(3) action representations across three standard continuous control algorithms, PPO, SAC, and TD3, under dense and sparse rewards. We compare how representations shape exploration, interact with entropy regularization, and affect training stability through empirical studies and analyze the implications of different projections for obtaining valid rotations from Euclidean network outputs. Across a suite of robotics benchmarks, we quantify the practical impact of these choices and distill simple, implementation-ready guidelines for selecting and using rotation actions. Our results highlight that representation-induced geometry strongly influences exploration and optimization and show that representing actions as tangent vectors in the local frame yields the most reliable results across algorithms. The project webpage and code are available at amacati.github.io/so3_primer.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。