arXiv:2605.04185cs.LGcs.RO2026-05

提出新方法让机器人运动更安全高效,约束匹配度提升50%。

Constraint-Enhanced Reinforcement Learning Based on Dynamic Decoupled Spherical Radial Squashing

论文配图:Constraint-Enhanced Reinforcement Learning Based on Dynamic Decoupled Spherical Radial Squashing
图 1 · 摘自论文原文
  • 为每个关节动态计算独立适应的约束半径,精准匹配实际运动限制。
  • 在模拟中实现零违规且任务回报达无约束上限,覆盖范围提升30%-50%。
  • 适合需要高精度物理控制的机器人系统,直接从硬件参数生成安全策略。

将强化学习策略部署于物理机器人时,执行器速率约束——即每个关节每控制步的最大运动速度——不可避免。由于电机惯性、功率带宽和传动刚度差异,各关节约束存在显著异质性。现有方法在几何上处理不当:动作增量空间中的可行区域为高维盒子,但二次规划投影与球形参数化方法施加各向同性球形约束,随异质性增加导致真实可行集被指数级低估。本文提出动态解耦球形径向挤压(DD-SRad),通过为每个执行器独立计算位置自适应半径,实现与真实单关节可行区域的紧密对齐。该方法以概率1满足每步硬约束,训练全程保持良好条件梯度,并支持零运行时开销的精确策略梯度反传。MuJoCo基准测试显示,任务回报达到无约束上界且零违规,约束空间覆盖率相较球形基线提升30%–50%。基于Unitree H1和G1人形机器人的高保真IsaacLab仿真验证了从官方关节规格直接参数化端到端最优策略的可行性,确立了从硬件数据手册到安全部署的系统路径。

原文摘要 · Abstract (English)

When deploying reinforcement learning policies to physical robots, actuator rate constraints -- hard limits on how fast each joint can move per control step -- are unavoidable. These limits vary substantially across joints due to differences in motor inertia, power bandwidth, and transmission stiffness, creating pronounced heterogeneity that existing methods fail to handle geometrically: the per-joint feasible region forms a high-dimensional box in action-increment space, yet QP projection and spherical parameterization methods impose isotropic ball-shaped constraints, exponentially under-covering the true feasible set as heterogeneity grows. This paper proposes Dynamic Decoupled Spherical Radial Squashing (DD-SRad), which resolves this mismatch by computing a position-adaptive radius independently for each actuator, achieving tight alignment with the true per-joint feasible region. DD-SRad satisfies per-step hard constraints with probability~1, preserves well-conditioned gradients throughout training, and admits exact policy gradient backpropagation with zero runtime solver overhead. MuJoCo benchmark experiments demonstrate the highest task return at zero constraint violation -- matching the unconstrained upper bound -- with 30%--50% improvement in constraint-space coverage over spherical baselines. High-fidelity IsaacLab simulations with Unitree H1 and G1 humanoid robots confirm end-to-end optimality parameterized directly from official joint specifications, validating a systematic pathway from hardware datasheets to safe deployment.

强化学习机器人控制约束优化安全部署

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。