用合成数据训练的模型，能零样本预测混沌系统
Panda: A pretrained forecast model for chaotic dynamics
- 基于演化算法生成2万+混沌系统数据，用注意力机制建模
- 仅在模拟数据上训练，却能准确预测真实实验时间序列
- 适合研究混沌系统、预训练模型或数学物理的学者
混沌系统对微小误差极度敏感，给基于数据的现实动力系统（如流体流动或神经活动）建模带来挑战。以往方法要么针对单个时间序列定制模型，要么在大规模时间序列数据库上训练基础模型但缺乏内在动力结构。受动力系统理论启发，我们提出Panda：用于非线性动力学的分块注意力模型。我们在一个新构建的可扩展合成数据集上训练Panda，该数据集包含2×10⁴个通过演化算法发现的混沌动力系统。模型仅在模拟数据上训练，却展现出涌现特性：对未见混沌系统实现零样本预测，保持短期精度与分布特性；注意力头中出现非线性共振模式；并有效预测真实世界实验时间序列。尽管训练仅使用低维常微分方程，Panda仍自发具备预测偏微分方程的能力，无需重新训练。我们还揭示了微分方程的神经缩放定律，凸显预训练模型在抽象数学领域（如非线性动力学）中的潜力。
原文摘要 · Abstract (English)
Chaotic systems are intrinsically sensitive to small errors, challenging efforts to construct predictive data-driven models of real-world dynamical systems such as fluid flows or neuronal activity. Prior efforts comprise either specialized models trained on individual time series, or foundation models trained on vast time series databases with little underlying dynamical structure. Motivated by dynamical systems theory, we present Panda, Patched Attention for Nonlinear DynAmics. We train Panda on a novel synthetic, extensible dataset of $2 \times 10^4$ chaotic dynamical systems that we discover using an evolutionary algorithm. Trained purely on simulated data, Panda exhibits emergent properties: zero-shot forecasting of unseen chaotic systems preserving both short-term accuracy and distributional measures, nonlinear resonance patterns in attention heads, and effective prediction of real-world experimental time series. Despite having been trained only on low-dimensional ordinary differential equations, Panda spontaneously develops the ability to predict partial differential equations without retraining. We also demonstrate a neural scaling law for differential equations, underscoring the potential of pre-trained models for probing abstract mathematical domains like nonlinear dynamics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。