用大模型自动融合物理知识,让符号回归更准更抗噪。
Knowledge Integration for Physics-informed Symbolic Regression Using Pre-trained Large Language Models
- 将大模型嵌入损失函数,自动评估方程的物理合理性。
- 在三种物理系统上均提升噪声下的方程重建精度。
- 适合非专家快速应用,降低符号回归的使用门槛。
符号回归(SR)是自动化科学发现的强大工具,可从实验数据中推导控制方程。将领域知识融入SR能提升发现方程的泛化性与实用性。物理信息符号回归(PiSR)通过引入领域知识解决此问题,但现有方法常需专用公式和手动特征工程,仅限领域专家使用。本文利用预训练大语言模型(LLM)实现PiSR中的知识集成。借助在海量科学文献上训练的LLM的上下文理解能力,自动整合领域知识,减少人工干预,使过程更普适。具体将LLM嵌入SR损失函数,增加其对生成方程的评价项。我们在三种SR算法(DEAP、gplearn、PySR)和三个LLM(Falcon、Mistral、LLama 2)下,针对三个物理动力学系统(自由落体、简谐运动、电磁波)进行评估。结果表明,引入LLM能持续提升从数据中重建物理动态的能力,增强模型对噪声和复杂性的鲁棒性。进一步分析提示工程影响,发现更丰富的提示显著提升性能。
原文摘要 · Abstract (English)
Symbolic regression (SR) has emerged as a powerful tool for automated scientific discovery, enabling the derivation of governing equations from experimental data. A growing body of work illustrates the promise of integrating domain knowledge into the SR to improve the discovered equation's generality and usefulness. Physics-informed SR (PiSR) addresses this by incorporating domain knowledge, but current methods often require specialized formulations and manual feature engineering, limiting their adaptability only to domain experts. In this study, we leverage pre-trained Large Language Models (LLMs) to facilitate knowledge integration in PiSR. By harnessing the contextual understanding of LLMs trained on vast scientific literature, we aim to automate the incorporation of domain knowledge, reducing the need for manual intervention and making the process more accessible to a broader range of scientific problems. Namely, the LLM is integrated into the SR's loss function, adding a term of the LLM's evaluation of the SR's produced equation. We extensively evaluate our method using three SR algorithms (DEAP, gplearn, and PySR) and three pre-trained LLMs (Falcon, Mistral, and LLama 2) across three physical dynamics (dropping ball, simple harmonic motion, and electromagnetic wave). The results demonstrate that LLM integration consistently improves the reconstruction of physical dynamics from data, enhancing the robustness of SR models to noise and complexity. We further explore the impact of prompt engineering, finding that more informative prompts significantly improve performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。