Tadpole用自编码器构建3D偏微分方程基础模型,支持高效在线训练与多任务应用。
Tadpole: Autoencoders as Foundation Models for 3D PDEs with Online Learning

- 基于在线生成数据的自编码器预训练,实现大规模无存储开销训练。
- 仅用少量可训练参数即可精准建模动态过程,支持多任务迁移。
- 适合需要快速适配新物理系统的科研与工程场景。
我们提出Tadpole,一种面向三维偏微分方程(PDEs)的新颖基础模型,解决迁移性、高维扩展性和多功能性挑战。Tadpole在由高效在线数据生成框架生成的合成3D PDE数据上进行自编码器预训练,实现无存储与I/O开销的大规模多样训练,等效训练数据量达数百TB。通过编码单通道空间局部区域,Tadpole学习跨异构物理系统(含不同状态变量数与空间分辨率)的丰富可迁移表征。尽管仅以自编码器形式预训练,仍可高效应用于重建以外的下游任务,如动力学学习与生成建模。针对动力学学习,我们提出一种参数高效的微调策略,结合低秩适配、隐空间变换与重引入的跳跃连接,在极小可训练参数量下实现精准时间演化建模。Tadpole在多种下游任务中表现出色,凸显其作为3D PDE学习基础模型的通用性与有效性。源代码与预训练权重已开源:https://github.com/tum-pbs/tadpole
原文摘要 · Abstract (English)
We introduce Tadpole, a novel foundation model for three-dimensional partial differential equations (PDEs) that addresses key challenges in transferability, scalability to high dimensionality, and multi-functionality. Tadpole is pre-trained as an autoencoder on synthetic 3D PDE data generated by an efficient online data-generation framework. This enables large-scale, diverse training without storage or I/O overhead, demonstrated by scaling to an equivalent of hundreds of terabytes of training data. By autoencoding single-channel spatial crops, Tadpole learns rich and transferable representations across heterogeneous physical systems with varying numbers of state variables and spatial resolutions. Although pre-trained solely as an autoencoder, Tadpole can be efficiently applied for multiple downstream tasks beyond reconstruction, including dynamics learning and generative modeling. For dynamics learning, we propose a novel parameter-efficient fine-tuning strategy that integrates low-rank adaptation, latent-space transformations, and reintroduced skip connections, achieving accurate temporal modeling with a minimal number of trainable parameters. Tadpole demonstrates strong fine-tuning performance across various downstream tasks, highlighting its versatility and effectiveness as a foundation model for 3D PDE learning. Source code and pre-trained weights of Tadpole are available at https://github.com/tum-pbs/tadpole
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。