arXiv:2602.01009cs.LGcs.AI2026-02被引 1

用线性近似加速微分方程计算,让大模型高效学习物理系统动态规律。

LASS-ODE: Scaling ODE Computations to Connect Foundation Models with Dynamical Physical Systems

  • 用局部线性微分方程表示法替代非线性积分,提升计算效率。
  • 在40GB轨迹数据上预训练,实现跨系统的零样本泛化能力。
  • 引入共享结构枢纽机制,增强多系统间知识迁移效果。

基础模型已变革语言、视觉与时间序列分析,但对物理系统动态预测进展有限。主要面临两大挑战:(i) 物理-计算可扩展性——物理约束学习虽能强化正则,但其计算(如常微分方程求解)难以扩展至大规模系统;(ii) 知识共享效率——注意力机制通常局限于单个系统内,难以提取跨系统共享的微分方程结构。本文提出,无需昂贵的非线性积分即可保证微分方程一致性:采用逐标记的局部线性微分方程表示,在保持物理保真度的同时,可扩展至基础模型规模。为此设计了尊重局部线性微分演化的新标记表示,显著加速积分过程并准确逼近局部数据流形。其次,引入一种简单而有效的跨系统注意力机制,通过共享结构枢纽(CSH)存储公共标记并聚合系统间知识。由此构建的LASS-ODE(Large-Scale Small ODE)模型在40GB微分方程轨迹数据集上预训练,实现强领域内性能、跨多样化微分方程系统的零样本泛化,并可通过微调进一步提升。

原文摘要 · Abstract (English)

Foundation models have transformed language, vision, and time series data analysis, yet progress on dynamic predictions for physical systems remains limited. Given the complexity of physical constraints, two challenges stand out. $(i)$ Physics-computation scalability: physics-informed learning can enforce physical regularization, but its computation (e.g., ODE integration) does not scale to extensive systems. $(ii)$ Knowledge-sharing efficiency: the attention mechanism is primarily computed within each system, which limits the extraction of shared ODE structures across systems. We show that enforcing ODE consistency does not require expensive nonlinear integration: a token-wise locally linear ODE representation preserves physical fidelity while scaling to foundation-model regimes. Thus, we propose novel token representations that respect locally linear ODE evolution. Such linearity substantially accelerates integration while accurately approximating the local data manifold. Second, we introduce a simple yet effective inter-system attention that augments attention with a common structure hub (CSH) that stores shared tokens and aggregates knowledge across systems. The resulting model, termed LASS-ODE (\underline{LA}rge-\underline{S}cale \underline{S}mall \underline{ODE}), is pretrained on our $40$GB ODE trajectory collections to enable strong in-domain performance, zero-shot generalization across diverse ODE systems, and additional improvements through fine-tuning.

微分方程大模型物理建模知识迁移

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。