用大模型发现抽象概念，提升符号回归效率和准确性
Symbolic Regression with a Learned Concept Library
- 通过零样本调用大模型，自动构建可复用的抽象概念库
- 在费曼方程和合成任务上超越现有最先进方法
- 可发现大模型的新缩放定律，适合研究者与算法工程师
我们提出一种新型符号回归（SR）方法，旨在寻找能最佳解释数据集的简洁程序化假设。传统方法多采用遗传算法；我们通过引入抽象文本概念库显著增强其性能。所提算法LaSR利用大语言模型（LLM）的零样本查询，发现并演化已知高性能假设中的核心概念。新假设通过标准进化步骤与LLM引导步骤结合生成，且基于已发现的概念进行条件化。发现的假设用于新一轮概念抽象与演化。我们在费曼方程（Feynman equations）这一经典SR基准以及一组合成任务上验证了该方法，结果表明LaSR显著优于多种基于深度学习与进化算法的先进方法。此外，我们展示了LaSR可发现一条新颖且强大的大模型缩放定律。
原文摘要 · Abstract (English)
We present a novel method for symbolic regression (SR), the task of searching for compact programmatic hypotheses that best explain a dataset. The problem is commonly solved using genetic algorithms; we show that we can enhance such methods by inducing a library of abstract textual concepts. Our algorithm, called LaSR, uses zero-shot queries to a large language model (LLM) to discover and evolve concepts occurring in known high-performing hypotheses. We discover new hypotheses using a mix of standard evolutionary steps and LLM-guided steps (obtained through zero-shot LLM queries) conditioned on discovered concepts. Once discovered, hypotheses are used in a new round of concept abstraction and evolution. We validate LaSR on the Feynman equations, a popular SR benchmark, as well as a set of synthetic tasks. On these benchmarks, LaSR substantially outperforms a variety of state-of-the-art SR approaches based on deep learning and evolutionary algorithms. Moreover, we show that LaSR can be used to discover a novel and powerful scaling law for LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。