arXiv:2607.10474cs.LGcs.AI2026-07

用可验证的物理奖励训练大模型,生成更精准的偏微分方程求解代码。

Reinforcement Learning with Verifiable Physics: Post-training LLMs with Continuous Rewards

论文配图:Reinforcement Learning with Verifiable Physics: Post-training LLMs with Continuous Rewards
图 1 · 摘自论文原文
  • 引入混合验证器:保证代码可运行,同时以连续物理指标评分解精度。
  • 在多类偏微分方程上训练单一策略,零样本迁移至未见方程。
  • 小模型经此训练可超越提示大模型,展现数值模式的组合能力。

偏微分方程(PDE)是科学与工程建模的基础,但构建可靠数值求解器仍需大量人力,依赖离散化方案、稳定性条件和边界处理等专业知识。近年研究将求解任务转化为大语言模型(LLM)的代码生成问题,但现有方法主要在推理阶段通过提示、调试与自精炼实现,未真正调整模型本身。与此同时,可验证奖励的强化学习(RL)已在代码与数学推理中兴起,但其验证器通常为二值信号:编译通过或测试通过。这类信号忽略了科学正确性的梯度特性——两个求解器均可执行,但解的精度可能相差数个数量级。本文提出 RLVP:基于可验证物理的强化学习框架,用于多类偏微分方程求解代码生成。该框架采用混合验证器:硬性程序有效性检查确保可执行性,连续物理奖励则评估函数空间精度与残差一致性。单一策略在涵盖双曲型、抛物型、椭圆型及不可压缩流系统的多种PDE家族上进行后训练。RLVP在多个PDE基准上优于预训练及仅监督训练基线,并展现出对未见PDE的零样本迁移能力。我们发现,经由RLVP后训练的小型模型,在同分布求解任务中可超越提示前沿模型的表现。训练后的策略表现出数值模式的组合性:能将训练中习得的差分格式、时间推进方案与边界处理原语重新组合,生成未见问题的求解器。

原文摘要 · Abstract (English)

Partial differential equations (PDEs) are foundational to modeling in science and engineering, but constructing reliable numerical solvers remains labor-intensive, demanding expert knowledge of discretization schemes, stability conditions, and boundary treatments. Recent work has begun to frame PDE solving as a code-generation task for large language models (LLMs), yet existing approaches operate primarily at inference time: relying on prompting, debugging, self-refinement, and test-time scaling rather than adapting the model itself. In parallel, reinforcement learning with verifiable rewards has emerged as a post-training paradigm for code and math reasoning, but its verifiers are typically binary: a compiler runs, or a test passes. Such signals discard the graded structure of scientific correctness, where two solvers may both execute and yet differ in solution accuracy by orders of magnitude. In this work, we introduce RLVP: Reinforcement Learning with Verifiable Physics, an RL post-training framework for multi-PDE solver code generation. RLVP addresses this verifiability gap with a hybrid verifier: hard program-validity checks ensure executability, while continuous physics rewards score function-space accuracy and PDE-residual consistency. A single policy is post-trained across diverse PDE families spanning hyperbolic, parabolic, elliptic, and incompressible-flow systems. RLVP improves over both pre-trained and supervised-only baselines on PDE benchmarks, and shows zero-shot improvement transfer to held-out PDEs. We show that a smaller LLM post-trained with RLVP can outperform prompting a frontier model on in-distribution PDE solver generation. The trained policy shows evidence of compositionality in numerical motifs: it recombines stencils, time-stepping schemes, and boundary-handling primitives learned from the PDEs used in training into generated solvers for unseen PDE problems.

强化学习偏微分方程代码生成物理驱动

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。