用物理量纲分析让强化学习策略零样本迁移，无需重训练
Zero-Shot Policy Transfer in Reinforcement Learning using Buckingham's Pi Theorem
- 基于皮定理构建无量纲空间，缩放输入输出实现零样本迁移
- 在动态相似场景下性能无损失，在非相似场景仍优于直接迁移
- 适合需快速适配新机器人或环境的工业应用，提升策略泛化能力
强化学习策略常难以泛化到具有不同物理参数的新机器人、任务或环境中，限制了其实际应用。本文提出一种基于伯金汉姆Pi定理的简单零样本迁移方法，通过将预训练策略的观测值和动作在无量纲空间中缩放，实现对新系统上下文的适应，无需重新训练。在三个复杂度递增的环境中评估：模拟摆、用于仿真到现实验证的物理摆，以及高维的HalfCheetah。结果表明，该方法在动态相似情境下性能无损失；在非相似情境下，其表现始终优于基线直接迁移方法，显著扩大了原策略的有效适用范围。研究证明，量纲分析是提升强化学习策略鲁棒性和泛化能力的强大且实用工具。
原文摘要 · Abstract (English)
Reinforcement learning (RL) policies often fail to generalize to new robots, tasks, or environments with different physical parameters, a challenge that limits their real-world applicability. This paper presents a simple, zero-shot transfer method based on Buckingham's Pi Theorem to address this limitation. The method adapts a pre-trained policy to new system contexts by scaling its inputs (observations) and outputs (actions) through a dimensionless space, requiring no retraining. The approach is evaluated against a naive transfer baseline across three environments of increasing complexity: a simulated pendulum, a physical pendulum for sim-to-real validation, and the high-dimensional HalfCheetah. Results demonstrate that the scaled transfer exhibits no loss of performance on dynamically similar contexts. Furthermore, on non-similar contexts, the scaled policy consistently outperforms the naive transfer, significantly expanding the volume of contexts where the original policy remains effective. These findings demonstrate that dimensional analysis provides a powerful and practical tool to enhance the robustness and generalization of RL policies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。