arXiv:2505.18602cs.NEcs.AI2025-05被引 3

用大模型自动设计符号回归的选代算子,效果超越专家设计。

LLM-Meta-SR: In-Context Learning for Evolving Selection Operators in Symbolic Regression

  • 让大模型基于语义设计选代算子,避免无效代码交换。
  • 在116个数据集上超越28种算法,性能达新高。
  • 适合研究自动化算法设计与符号回归的学者。

大语言模型(LLMs)虽已革新算法开发,但在符号回归领域应用仍有限。本文提出一种元学习框架,使LLM能自动设计进化符号回归算法中的选择算子。我们识别出现有基于LLM的算法演化技术的两大缺陷:缺乏语义引导和代码膨胀。语义缺失导致有效代码组件交换失效,而膨胀则引发冗余复杂度,阻碍进化学习或降低可解释性。为此,我们改进了基于LLM的元符号回归框架,引入语义感知的选择算子和膨胀控制机制,并在提示中嵌入领域知识,使LLM生成更高效且上下文相关的算子。实验结果表明,所设计算子在符号回归基准测试中优于九个专家设计基线,达到当前最佳性能;且能进一步提升一种顶尖符号回归算法,在116个回归数据集上优于28种符号回归及其他机器学习算法,证明大模型在符号回归算法设计上可超越人类专家。

原文摘要 · Abstract (English)

Large language models (LLMs) have revolutionized algorithm development, yet their application in symbolic regression, where algorithms automatically discover symbolic expressions from data, remains limited. In this paper, we propose a meta-learning framework that enables LLMs to automatically design selection operators for evolutionary symbolic regression algorithms. We first identify two key limitations in existing LLM-based algorithm evolution techniques: lack of semantic guidance and code bloat. The absence of semantic awareness can lead to ineffective exchange of useful code components, while bloat results in unnecessarily complex components; both can hinder evolutionary learning progress or reduce the interpretability of the designed algorithm. To address these issues, we enhance the LLM-based evolution framework for meta-symbolic regression with two key innovations: a complementary, semantics-aware selection operator and bloat control. Additionally, we embed domain knowledge into the prompt, enabling the LLM to generate more effective and contextually relevant selection operators. Our experimental results on symbolic regression benchmarks show that LLMs can devise selection operators that outperform nine expert-designed baselines, achieving state-of-the-art performance. Moreover, the evolved operator can further improve a state-of-the-art symbolic regression algorithm, achieving the best performance among 28 symbolic regression and other machine learning algorithms across 116 regression datasets. This demonstrates that LLMs can exceed expert-level algorithm design for symbolic regression.

符号回归大模型算法设计元学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。