用分块搜索空间提升大模型在复杂优化中的表现
Improving LLM-based Global Optimization with Search Space Partitioning
- 将搜索空间划分为多个子区域，以更好引导大模型采样
- 在标准测试中优于主流优化方法，且显著超越普通大模型策略
- 适合高维优化任务，无需领域知识即可高效工作
大型语言模型（LLMs）近年来被用作昂贵黑箱函数全局优化框架中的代理模型和候选生成器。尽管效果显著，但基于LLM的方法在高维搜索空间或缺乏领域先验时，常产生稀疏或无信息的建议。为此，我们提出HOLLM，一种新型全局优化算法，通过将搜索空间划分为有潜力的子区域，提升LLM驱动的采样效率。每个子区域作为“元臂”，通过类强化学习的评分机制选择，有效平衡探索与利用。在选定的子区域内，LLM生成高质量候选点，无需显式领域知识。在标准优化基准上的实证评估表明，HOLLM在多数情况下匹配或超越领先优化方法，且显著优于全局的LLM采样策略。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have recently emerged as effective surrogate models and candidate generators within global optimization frameworks for expensive blackbox functions. Despite promising results, LLM-based methods often struggle in high-dimensional search spaces or when lacking domain-specific priors, leading to sparse or uninformative suggestions. To overcome these limitations, we propose HOLLM, a novel global optimization algorithm that enhances LLM-driven sampling by partitioning the search space into promising subregions. Each subregion acts as a ``meta-arm'' selected via a bandit-inspired scoring mechanism that effectively balances exploration and exploitation. Within each selected subregion, an LLM then proposes high-quality candidate points, without any explicit domain knowledge. Empirical evaluation on standard optimization benchmarks shows that HOLLM consistently matches or surpasses leading global optimization methods, while substantially outperforming global LLM-based sampling strategies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。