arXiv:2509.21670cs.CVcs.AI2025-09被引 10

MORPH可统一处理多模态、跨维度的科学数据,实现高效物理方程预测。

MORPH: PDE Foundation Models with Arbitrary Data Modality

  • 采用卷积视觉变压器架构,融合局部交互与跨场信息传播机制。
  • 在多种异构PDE数据集上预训练,微调后性能超越从零训练模型。
  • 适合需要处理多模态、多尺度科学数据的研究者使用。

我们提出MORPH,一种对数据模态无感的自回归偏微分方程(PDE)基础模型。MORPH基于卷积视觉变压器骨干网络,可无缝处理不同维度(1D–3D)、分辨率各异且包含标量与矢量分量的异构时空数据集。其架构结合:(i) 分量级卷积,联合处理标量与矢量通道以捕捉局部相互作用;(ii) 跨场交叉注意力,建模并选择性传播不同物理场间信息;(iii) 轴向注意力,沿空间与时间轴分解全时空自注意力,降低计算开销同时保持表达能力。我们在多样化的异构PDE数据集上预训练多个模型变体,并评估其在下游预测任务中的迁移能力。通过全模型微调和参数高效的低秩适配器,MORPH表现优于从头训练模型。在广泛评估中,MORPH达到或超过强基线与最新先进模型。这些能力为从异构、多模态科学观测中学习提供了灵活而强大的骨干网络,推动可扩展、数据高效的科学机器学习发展。源码、数据集与模型已公开于https://github.com/lanl/MORPH。

原文摘要 · Abstract (English)

We introduce MORPH, a modality-agnostic, autoregressive foundation model for partial differential equations (PDEs). MORPH is built on a convolutional vision transformer backbone that seamlessly handles heterogeneous spatiotemporal datasets of varying data modality (1D--3D) at different resolutions, and multiple fields with mixed scalar and vector components. The architecture combines (i) component-wise convolution, which jointly processes scalar and vector channels to capture local interactions, (ii) inter-field cross-attention, which models and selectively propagates information between different physical fields, (iii) axial attentions, which factorize full spatiotemporal self-attention along individual spatial and temporal axes to reduce computational burden while retaining expressivity. We pretrain multiple model variants on a diverse collection of heterogeneous PDE datasets and evaluate transfer to a range of downstream prediction tasks. Using both full-model fine-tuning and parameter-efficient low-rank adapters, MORPH outperforms models trained from scratch. Across extensive evaluations, MORPH matches or surpasses strong baselines and recent state-of-the-art models. Collectively, these capabilities present a flexible and powerful backbone for learning from the heterogeneous and multimodal nature of scientific observations, charting a path toward scalable and data-efficient scientific machine learning. The source code, datasets, and models are publicly available at https://github.com/lanl/MORPH.

PDE建模多模态基础模型科学机器学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。