arXiv:2410.17448cs.CLcs.AI2024-10被引 8

用大模型自动发现科学方程,结合自然语言提示提升推理能力。

In Context Learning and Reasoning for Symbolic Regression with Large Language Models

  • 通过思维链提示让模型分析数据与科学背景,逐步生成方程。
  • 在朗缪尔吸附等3个案例中成功复现已知方程,尼古拉兹数据集获新解。
  • 结合领域知识提示可显著提高生成有意义方程的概率。

大型语言模型(LLMs)在未显式训练的任务中表现出色。本文探索其在符号回归中的潜力——从数据中寻找简洁准确的数学表达式。我们使用GPT-4和GPT-4o模型,通过提示生成候选表达式,并利用外部Python工具优化评估,将结果反馈给模型以迭代改进。采用思维链提示策略,指导模型分析数据、已有表达式及科学背景(自然语言描述)。在朗缪尔与双位点朗缪尔吸附模型的复现,以及无已知目标方程的尼古拉兹粗糙管道流动数据集上,两者均成功重构方程,使用草稿板和科学上下文时表现更优。GPT-4o在双位点朗缪尔与尼古拉兹数据集中展现出更强的数据模式推理能力。结果表明,合理提示可提升模型性能,自然语言接口使理论与数据融合更简便。通过提示引入基于背景知识的数学约束,显著增加模型生成有效方程的频率。尽管在复杂目标方程上仍不及专业符号回归程序,但该方法能遵循指令并融入自然语言科学语境,持续迭代优化解决方案。

原文摘要 · Abstract (English)

Large Language Models (LLMs) are transformer-based machine learning models that have shown remarkable performance in tasks for which they were not explicitly trained. Here, we explore the potential of LLMs to perform symbolic regression -- a machine-learning method for finding simple and accurate equations from datasets. We prompt GPT-4 and GPT-4o models to suggest expressions from data, which are then optimized and evaluated using external Python tools. These results are fed back to the LLMs, which propose improved expressions while optimizing for complexity and loss. Using chain-of-thought prompting, we instruct the models to analyze data, prior expressions, and the scientific context (expressed in natural language) for each problem before generating new expressions. We evaluated the workflow in rediscovery of Langmuir and dual-site Langmuir's model for adsorption, along with Nikuradse's dataset on flow in rough pipes, which does not have a known target model equation. Both the GPT-4 and GPT-4o models successfully rediscovered equations, with better performance when using a scratchpad and considering scientific context. GPT-4o model demonstrated improved reasoning with data patterns, particularly evident in the dual-site Langmuir and Nikuradse dataset. We demonstrate how strategic prompting improves the model's performance and how the natural language interface simplifies integrating theory with data. We also applied symbolic mathematical constraints based on the background knowledge of data via prompts and found that LLMs generate meaningful equations more frequently. Although this approach does not outperform established SR programs where target equations are more complex, LLMs can nonetheless iterate toward improved solutions while following instructions and incorporating scientific context in natural language.

符号回归大模型科学发现提示工程

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。