让机器人在运行中突然遭遇变化时快速恢复,不需重新训练。
Residual Control for Fast Recovery from Dynamics Shifts
- 用受控的残差通道修正动作,保持原策略不变
- 在四种机器人上平均恢复时间缩短87%
- 适合需要稳定实时响应的机器人场景
在真实环境中运行的机器人不可避免地会遇到未观测到的动力学变化,如执行器退化、质量分布改变或接触条件变化。当这些变化发生在任务中途时,即使经过训练的策略具有局部稳定性,也会出现显著的性能下降。尽管输入-状态稳定性可保证状态偏差有界,但无法确保任务性能的快速恢复。本文提出一种在冻结策略参数下实现推理时恢复的方法,将适应过程建模为围绕一个名义稳定控制器的受限扰动塑造。所提出的稳定性对齐残差控制架构中,强化学习策略在名义动力学下训练并部署时保持固定,适应仅通过一个有界加性残差通道实现。稳定性对齐门(SAG)通过幅度约束、与名义动作的方向一致性、性能条件激活和自适应增益调节来调控修正力度。这些机制在保持名义闭环结构的同时,无需重新训练或特权扰动信息即可快速补偿未观测到的动力学变化。在包括执行器退化、质量变化和接触改变在内的多种中期扰动下,该方法相比冻结策略和在线自适应基线表现出更短的恢复时间,同时维持接近名义的稳态性能。相对于冻结的SAC策略,Go1四足机器人恢复时间减少87%,Cassie双足机器人减少48%,H1人形机器人减少30%,Scout轮式平台减少20%。
原文摘要 · Abstract (English)
Robotic systems operating in real-world environments inevitably encounter unobserved dynamics shifts during continuous execution, including changes in actuation, mass distribution, or contact conditions. When such shifts occur mid-episode, even locally stabilizing learned policies can experience substantial transient performance degradation. While input-to-state stability guarantees bounded state deviation, it does not ensure rapid restoration of task-level performance. We address inference-time recovery under frozen policy parameters by casting adaptation as constrained disturbance shaping around a nominal stabilizing controller. We propose a stability-aligned residual control architecture in which a reinforcement learning policy trained under nominal dynamics remains fixed at deployment, and adaptation occurs exclusively through a bounded additive residual channel. A Stability Alignment Gate (SAG) regulates corrective authority through magnitude constraints, directional coherence with the nominal action, performance-conditioned activation, and adaptive gain modulation. These mechanisms preserve the nominal closed-loop structure while enabling rapid compensation for unobserved dynamics shifts without retraining or privileged disturbance information. Across mid-episode perturbations including actuator degradation, mass variation, and contact changes, the proposed method consistently reduces recovery time relative to frozen and online-adaptation baselines while maintaining near-nominal steady-state performance. Recovery time is reduced by \textbf{87\%} on the Go1 quadruped, \textbf{48\%} on the Cassie biped, \textbf{30\%} on the H1 humanoid, and \textbf{20\%} on the Scout wheeled platform on average across evaluated conditions relative to a frozen SAC policy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。