用大模型理解文字描述，让物理方程求解更准更快
Explain Like I'm Five: Using LLMs to Improve PDE Surrogate Models with Text
- 用LLM将方程、边界条件等信息转为文本输入，融合到PDE求解器中
- 仅凭初始条件，也能在多个复杂方程上实现高精度预测
- 适合想简化物理建模流程的研究者和工程师
求解偏微分方程（PDE）在科学与工程中无处不在。计算复杂度高及数值求解器编写困难，推动了数据驱动的机器学习方法发展，以快速生成解。近年来大型语言模型（LLMs）的兴起，使文本可轻松融入多模态机器学习模型，从而通过文字整合边界条件和控制方程等系统信息。本文探索使用预训练LLM将不同量的已知系统信息注入PDE学习过程。以FactFormer为基准模型，添加多模态模块融合数值与文本信息。在二维热传导、伯格斯、纳维-斯托克斯和浅水方程数据集上，对比了句子级嵌入、词级嵌入与标准分词器的表现。结果表明，预训练LLM能有效利用系统信息的文本描述，在仅提供初始条件的情况下实现高精度预测。
原文摘要 · Abstract (English)
Solving Partial Differential Equations (PDEs) is ubiquitous in science and engineering. Computational complexity and difficulty in writing numerical solvers has motivated the development of data-driven machine learning techniques to generate solutions quickly. The recent rise in popularity of Large Language Models (LLMs) has enabled easy integration of text in multimodal machine learning models, allowing easy integration of additional system information such as boundary conditions and governing equations through text. In this work, we explore using pretrained LLMs to integrate various amounts of known system information into PDE learning. Using FactFormer as our testing backbone, we add a multimodal block to fuse numerical and textual information. We compare sentence-level embeddings, word-level embeddings, and a standard tokenizer across 2D Heat, Burgers, Navier-Stokes, and Shallow-Water data sets. These challenging benchmarks show that pretrained LLMs are able to utilize text descriptions of system information and enable accurate prediction using only initial conditions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。