arXiv:2503.08301cs.LGcs.AI2025-03被引 5

用大模型做元代理,高效解决多任务优化问题

Large Language Model as Meta-Surrogate for Data-Driven Many-Task Optimization: A Proof-of-Principle Study

  • 用大模型统一建模任务元数据与输入输出,实现跨任务知识共享
  • 在未见过维度的问题上实现零样本预测,展现强泛化能力
  • 适合需要快速适应新优化任务的研究者和工程应用

在多任务优化中,代理模型可缓解重复评估适应度带来的计算负担。本文提出一种基于大语言模型(LLM)的新型元代理框架,利用其知识迁移能力和涌现特性,构建统一的多任务适应度预测模型。通过定义包含元数据的通用模型,对元数据与决策变量进行联合建模,实现跨任务高效知识共享与新任务自适应。将适应度预测建模为条件概率估计,采用统一的令牌序列表示任务元数据、输入与输出,借助共享令牌嵌入实现任务间高效知识传递,并通过多任务训练捕捉复杂任务依赖关系。实验表明该模型具备显著涌现泛化能力,可在未见维度问题上实现零样本性能。集成至进化转移优化(ETO)后,支持代理层与个体层双重知识迁移,提升优化效率与鲁棒性。本工作为大模型在代理建模中的应用奠定新基础,提供适用于多任务优化的通用解决方案。

原文摘要 · Abstract (English)

In many-task optimization scenarios, surrogate models are valuable for mitigating the computational burden of repeated fitness evaluations across tasks. This study proposes a novel meta-surrogate framework to assist many-task optimization, by leveraging the knowledge transfer strengths and emergent capabilities of large language models (LLMs). We formulate a unified framework for many-task fitness prediction, by defining a universal model with metadata to fit a group of problems. Fitness prediction is performed on metadata and decision variables, enabling efficient knowledge sharing across tasks and adaptability to new tasks. The LLM-based meta-surrogate treats fitness prediction as conditional probability estimation, employing a unified token sequence representation for task metadata, inputs, and outputs. This approach facilitates efficient inter-task knowledge sharing through shared token embeddings and captures complex task dependencies via multi-task model training. Experimental results demonstrate the model's emergent generalization ability, including zero-shot performance on problems with unseen dimensions. When integrated into evolutionary transfer optimization (ETO), our framework supports dual-level knowledge transfer -- at both the surrogate and individual levels -- enhancing optimization efficiency and robustness. This work establishes a novel foundation for applying LLMs in surrogate modeling, offering a versatile solution for many-task optimization.

大模型多任务优化代理模型零样本

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。