用纯合成数据训练动态系统基础模型，实现跨系统的强泛化能力。
On Foundation Models for Dynamical Systems from Purely Synthetic Data
- 基于核空间采样生成合成动态数据，用Transformer预训练基础模型。
- 在仿真与真实硬件上验证，对不同系统预测表现优于专用模型。
- 可高效微调适配新系统，适合需要快速部署的控制场景。
基础模型在自然语言处理和计算机视觉等领域展现出卓越的泛化、数据效率和鲁棒性。本文探索其在控制领域的可行性。现有大规模预训练依赖互联网级数据集，但动态系统领域尚无此类资源。为此，我们仅使用合成数据预训练一个基于Transformer的基础模型，并从再生核希尔伯特空间中采样动态函数。预训练模型在多种动态系统上表现出优异的预测泛化能力，已在仿真和真实硬件实验中验证，包括倒立摆和Furuta摆系统。此外，模型可通过微调有效适应新系统，进一步提升性能。结果表明，该方法可实现优于专用模型的泛化性、数据效率与鲁棒性。
原文摘要 · Abstract (English)
Foundation models have demonstrated remarkable generalization, data efficiency, and robustness properties across various domains. In this paper, we explore the feasibility of foundation models for applications in the control domain. The success of these models is enabled by large-scale pretaining on Internet-scale datasets. These are available in fields like natural language processing and computer vision, but do not exist for dynamical systems. We address this challenge by pretraining a transformer-based foundation model exclusively on synthetic data and propose to sample dynamics functions from a reproducing kernel Hilbert space. Our pretrained model generalizes for prediction tasks across different dynamical systems, which we validate in simulation and hardware experiments, including cart-pole and Furuta pendulum setups. Additionally, the model can be fine-tuned effectively to new systems to increase performance even further. Our results demonstrate the feasibility of foundation models for dynamical systems that outperform specialist models in terms of generalization, data efficiency, and robustness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。