arXiv:2506.15700cs.LGcs.AI2025-06中稿 · IEEE Transactions …

用收缩度量指导强化学习,让非线性控制更稳定且抗模型误差。

Contraction-Aware Reinforcement Learning for Nonlinear Control with Statistical Robustness

  • 将收缩度量嵌入强化学习,实现稳定性与最优性的兼顾
  • 在模拟和真实机器人上均提升轨迹跟踪性能,对模型误差更鲁棒
  • 适合需要高可靠性非线性控制的机器人系统研发

控制收缩度量(CCMs)——即在黎曼度量下使闭环系统增量指数稳定的度量——为非线性路径跟踪问题提供了构造收缩策略的可实现框架。然而,虽然合成策略能保证点态满足CCM条件,却难以确保在瞬态和稳态阶段累积轨迹误差最小化。此外,这类策略的短视特性在使用近似动力学建模时可能加剧学习偏差。为此,本文提出将CCMs融入强化学习(RL)。CCMs为学习提供动态信息反馈,使策略具备稳定性保障(即收缩感知),而RL则在近似动力学下最小化累积跟踪误差。基于预训练动力学模型,我们的算法——收缩感知强化学习(CARL)——同时学习生成CCMs并优化由这些度量定义奖励的策略。实验表明,相比基线方法,CARL在仿真与真实机器人实验中均显著提升路径跟踪性能,并对近似动力学误差表现出更强鲁棒性。我们还提供了将CCMs融入RL的理论依据。代码已开源:https://github.com/Mgineer117/CARL,真实实验视频见:https://youtu.be/sOJ4hulbop0。

原文摘要 · Abstract (English)

Control contraction metrics (CCMs)-defined by Riemannian metrics under which a closed-loop system is incrementally exponentially stable-offer a constructive framework for synthesizing contracting policies in nonlinear path-tracking problems. However, while the synthesized policies ensure pointwise satisfaction of the CCM conditions, they may not ensure long-term optimality (i.e., minimizing cumulative trajectory-level tracking error) over both transient and steady-state regimes. Furthermore, the myopic nature of these policies could also make them more susceptible to learning biases when approximate dynamics are used to formulate CCMs. To address these issues, we propose to integrate CCMs into reinforcement learning (RL). CCMs provide dynamics-informed feedback for learning a policy that has a stability guarantee-i.e., is contraction-aware-while RL provides a framework for minimizing cumulative tracking error under approximate dynamics. Given a pretrained dynamics model, our algorithm, contraction-aware RL (CARL), simultaneously learns to generate CCMs and optimize a policy for rewards defined by those CCMs. We demonstrate that CARL enhances path-tracking performance and is robust to errors in approximated dynamics compared to relevant baselines in both simulated and real-world robot experiments. We also provide theoretical rationale for integrating CCMs into RL. Our code is available at https://github.com/Mgineer117/CARL, and a video of our real-world robot experiments can be found at https://youtu.be/sOJ4hulbop0.

强化学习非线性控制稳定性保障

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。