arXiv:2410.03437cs.LG2024-10ICML被引 17

用上下文学习解参数化偏微分方程,无需反向传播优化。

Zebra: In-Context Generative Pretraining for Solving Parametric PDEs

  • 基于上下文信息动态适配新任务,无需梯度更新。
  • 在多种复杂PDE场景中表现优于现有方法,支持不确定性量化。
  • 生成式设计可预测新轨迹,适合需要灵活适应的科学计算场景。

求解时变参数化偏微分方程对数据驱动方法构成挑战,因模型需适应系数、源项和初值等参数变化。当前最优神经代理通过梯度优化与元学习隐式编码多类动态,但推理复杂度高。受大语言模型上下文学习启发,我们提出Zebra——一种生成式自回归变换器,可在推理时不依赖梯度调整。通过在预训练和推理阶段利用上下文信息,Zebra通过输入包含示例轨迹的序列实现动态适配。作为生成模型,它能生成新轨迹并量化预测不确定性。我们在多种挑战性PDE场景中评估了Zebra,证明其具备良好适应性、鲁棒性及超越现有方法的性能。

原文摘要 · Abstract (English)

Solving time-dependent parametric partial differential equations (PDEs) is challenging for data-driven methods, as these models must adapt to variations in parameters such as coefficients, forcing terms, and initial conditions. State-of-the-art neural surrogates perform adaptation through gradient-based optimization and meta-learning to implicitly encode the variety of dynamics from observations. This often comes with increased inference complexity. Inspired by the in-context learning capabilities of large language models (LLMs), we introduce Zebra, a novel generative auto-regressive transformer designed to solve parametric PDEs without requiring gradient adaptation at inference. By leveraging in-context information during both pre-training and inference, Zebra dynamically adapts to new tasks by conditioning on input sequences that incorporate context example trajectories. As a generative model, Zebra can be used to generate new trajectories and allows quantifying the uncertainty of the predictions. We evaluate Zebra across a variety of challenging PDE scenarios, demonstrating its adaptability, robustness, and superior performance compared to existing approaches.

偏微分方程生成模型上下文学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。