arXiv:2605.14546cs.LG2026-05被引 2

发现神经PDE模型权重空间中的物理方向,实现无需训练的跨参数迁移。

Discovering Physical Directions in Weight Space: Composing Neural PDE Experts

论文配图:Discovering Physical Directions in Weight Space: Composing Neural PDE Experts
图 1 · 摘自论文原文
  • 通过微调两端物理态专家,揭示权重更新中可分离出共享适应与物理方向。
  • 提出CCM方法,在已知物理参数下直接推断并组合专家模型,提升外推性能。
  • 在反应-扩散、纳维-斯托克斯等系统上显著降低分布外误差,尤其适用于高难度外推场景。

神经算子的进展使偏微分方程(PDE)代理建模在大规模预训练和上下文自适应下更具可扩展性和迁移性。然而,当一个共享算子被微调以适应连续物理家族中的多个状态时,其权重更新是否仅形成孤立的特定状态专家,还是揭示了可重用的物理结构仍不明确。本文从共享家族起点出发,微调低、高参数端点专家,发现其权重更新可分解为共享适应部分和与底层物理参数对齐的方向。这一分离将端点专家重新解释为权重空间中局部物理方向的有限差分探测器,解释了为何静态平均能插值但会弱化端点特异性物理。基于此,提出校准条件合并(CCM)方法,一种后处理坐标读取机制,用于沿该物理方向组合神经PDE专家。给定物理元数据、校准坐标映射或短观测序列前缀,CCM可推断目标组合坐标,并部署单个合并检查点完成后续预测。在反应-扩散系统、黏度参数化的二维纳维-斯托克斯方程和径向溃坝动力学上评估,跨基准,CCM在外推情形下表现最优,相较家族起点分别降低分布外滚动误差54.2%、42.8%和13.8%。多尺度FNO、DPOT风格主干及消融实验进一步验证,端点微调并非任意检查点漂移,而是揭示了可校准的物理方向,支持训练零成本的跨PDE状态迁移。

原文摘要 · Abstract (English)

Recent advances in neural operators have made partial differential equation (PDE) surrogate modeling increasingly scalable and transferable through large-scale pretraining and in-context adaptation. However, after a shared operator is fine-tuned to multiple regimes within a continuous physical family, it remains unclear whether the resulting weight-space updates merely form isolated regime experts or reveal reusable physical structure. Starting from a shared family anchor, we fine-tune low- and high-regime endpoint experts and show that their updates can be separated into a family-shared adaptation and a direction aligned with the underlying physical parameter. This separation reinterprets endpoint experts as finite-difference probes of a local physical direction in weight space, explaining why static averaging can interpolate between regimes but attenuates endpoint-specific physics. Building on this perspective, we propose Calibration-Conditioned Merge (CCM), a post-hoc coordinate readout method for composing neural PDE experts along this physical direction. Given physical metadata, a calibrated coordinate mapping, or a short observed rollout prefix, CCM infers the target composition coordinate and deploys a single merged checkpoint for the remaining rollout. We evaluate CCM on the reaction--diffusion system, viscosity-parameterized two-dimensional Navier--Stokes equations, and radial dam-break dynamics. Across these benchmarks, CCM achieves its strongest gains in extrapolative regimes, reducing out-of-distribution rollout error relative to the family anchor by 54.2%, 42.8%, and 13.8%, respectively. Further experiments across FNO scales, a DPOT-style backbone, and ablations confirm that endpoint fine-tuning is not arbitrary checkpoint drift, but reveals a calibratable physical direction for training-free transfer across PDE regimes.

PDE建模神经算子迁移学习物理方向

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。