用大模型自动生成可通用的启发式算法,解决传统方法泛化差的问题。
Generalizable Heuristic Generation Through LLMs with Meta-Optimization
- 通过元优化框架让大模型自主生成多样启发式算子
- 在多种组合优化任务中达到顶尖性能,跨规模测试表现突出
- 适合对算法自动化和泛化能力有要求的研究者
利用大语言模型(LLMs)设计启发式算法已成为解决组合优化问题(COPs)的有前景方法。然而,现有方法通常依赖预设的进化计算(EC)启发式优化器和单任务训练策略,限制了启发式算法的多样性探索,并影响其泛化能力。为此,我们提出元优化启发式(MoH)框架,从优化器层面出发,基于元学习原理发现有效的启发式优化器。MoH利用大模型迭代优化一个元优化器,该优化器通过(自)调用方式自主构建多样化的启发式优化器,从而摆脱对预定义优化器的依赖。这些生成的优化器随后用于演化下游任务的启发式算法,实现更广泛的探索。此外,MoH采用多任务训练方案提升泛化能力。在经典组合优化问题上的实验表明,MoH能构建出高效且可解释的元优化器,在多种下游任务中表现优异,尤其在跨规模设置下效果显著。代码已开源:https://github.com/yiding-s/MoH。
原文摘要 · Abstract (English)
Heuristic design with large language models (LLMs) has emerged as a promising approach for tackling combinatorial optimization problems (COPs). However, existing approaches often rely on manually predefined evolutionary computation (EC) heuristic-optimizers and single-task training schemes, which may constrain the exploration of diverse heuristic algorithms and hinder the generalization of the resulting heuristics. To address these issues, we propose Meta-Optimization of Heuristics (MoH), a novel framework that operates at the optimizer level, discovering effective heuristic-optimizers through the principle of meta-learning. Specifically, MoH leverages LLMs to iteratively refine a meta-optimizer that autonomously constructs diverse heuristic-optimizers through (self-)invocation, thereby eliminating the reliance on a predefined EC heuristic-optimizer. These constructed heuristic-optimizers subsequently evolve heuristics for downstream tasks, enabling broader heuristic exploration. Moreover, MoH employs a multi-task training scheme to promote its generalization capability. Experiments on classic COPs demonstrate that MoH constructs an effective and interpretable meta-optimizer, achieving state-of-the-art performance across various downstream tasks, particularly in cross-size settings. Our code is available at: https://github.com/yiding-s/MoH.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。