让大模型生成的模拟代码不再只靠跑通来判断对错，而是检查是否真在解用户想要的物理方程。
Your Simulation Runs but Solves the Wrong Physics: PDE-Grounded Intent Verification for LLM-Generated Multiphysics Simulation Code
- 通过重构代码中的偏微分方程结构，与用户意图对比验证
- 在220个案例中，修复后意图契合度提升0.22至0.41
- 适合关注科学模拟正确性的研究人员和工程应用者
大模型生成的科学模拟代码常因执行成功而被误判为正确，但可能实际求解的是错误的物理规律。我们定义了‘理解-生成差距’，并以MOOSE框架为例，利用内核与边界条件的组合结构，可确定性重建编码的偏微分方程（PDE），并与用户意图进行比对。提出意图保真度评分（IFS），涵盖控制项、边界条件、初始条件、系数和时间格式等。基于IFS构建可迭代修正的闭环流程。在包含220个案例的MooseBench基准上测试，该方法显著提升平均IFS，尤其在直接生成表现差（IFS<0.7）的难题中提升最明显。部署审计显示，仅依赖执行成功的修复方案虽提高运行率，但仍有39%-40%的案例仍解错了物理问题，揭示执行与意图保真度是两类独立失败模式。在四种其他偏微分方程领域专用语言（UFL/FEniCS、FreeFEM、FiPy、Devito）的初步实验也表明该方法具有普适性。研究强调：科学模拟代码必须验证其数学结构是否匹配预期，不能仅凭运行成功。
原文摘要 · Abstract (English)
Execution-based evaluation of LLM-generated code implicitly treats successful execution as a proxy for correctness. In scientific simulation, this proxy is insufficient: a generated input file can run, mesh, and converge while encoding governing equations that differ from the user's intent. We call this mismatch between intended physics and generated code the comprehension-generation gap. We instantiate this in MOOSE, where Kernel and BC objects map compositionally to weak-form residual terms, enabling deterministic reconstruction of the encoded PDE and comparison against an intended contract. We formalize this comparison as the Intent Fidelity Score (IFS), a structural metric covering governing terms, BCs, ICs, coefficients, and time scheme. Building on IFS, we develop a PDE-grounded refinement loop that uses deterministic violation reports to correct generated code iteratively. We evaluate on MooseBench, a 220-case multiphysics benchmark with PDE-level ground truth released with this work. On this benchmark, our method consistently improves mean IFS over direct generation, with gains concentrated on hard cases. On the subset where direct generation falls below IFS 0.7, refinement adds +0.22 to +0.41 absolute IFS. In the deployment audit, execution-only repair improves execution success while leaving 39-40% of all 220 cases runnable but still solving the wrong physics across the three main deployment-audit models, exposing executability and intent fidelity as separable failure modes. Static proof-of-concept experiments on four PDE-oriented DSLs (UFL/FEniCS, FreeFEM, FiPy, and Devito) suggest that the reconstruction-and-comparison pattern extends beyond MOOSE. These findings reinforce that executable simulation code should be verified against the mathematical structure it is intended to encode, not accepted on execution alone.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。