用语言模型统一建模多任务多目标优化,提升精度与收敛性。
Offline Multi-Task Multi-Objective Data-Driven Evolutionary Algorithm with Language Surrogate Model and Implicit Q-Learning
- 将多目标优化转为序列到序列建模,用大语言模型预测目标值。
- 在CEC2019上比基线更准,帮助进化算法更快收敛且逼近最优解。
- 适合处理复杂高维多目标优化问题的研究者与工程应用。
数据驱动的进化算法在解决高成本优化问题上表现出色,依赖于稳健的代理建模。然而,现有方法在具有多个子目标的复杂问题中仍受限于重复且繁琐的近似过程。为此,我们提出Q-MetaSur——一种可即插即用的代理建模方案,实现统一且泛化的代理学习。具体而言,我们在离线设置下研究多任务多目标优化(MTMOO)。关键设计包括:1)将目标近似转化为序列到序列建模,通过文本化标记表示MTMOO问题;2)引入基于大语言模型的代理模型,先编码一个MTMOO实例,再解码未见决策变量的目标值;3)提出两阶段离线训练策略,结合监督微调与强化学习精调,先利用离线数据集拟合已有知识,再通过强化学习提升模型泛化能力。在CEC2019基准上的大量实验表明,Q-MetaSur不仅在目标近似准确率上优于代表性基线,还使底层进化算法实现更优的优化收敛性和帕累托最优性。
原文摘要 · Abstract (English)
Data-driven evolutionary algorithms has shown surprising results in addressing expensive optimization problems through robust surrogate modeling. Though promising, existing surrogate modeling schemes may encounter limitations in complex optimization problems with many sub-objectives, which rely on repeated and tedious approximation. To address such technical gap, we propose Q-MetaSur as a plug-and-play surrogate modeling scheme capable of providing unified and generalized surrogate learning. Specifically, we consider multi-task-multi-objective optimization~(MTMOO) in offline setting. Several key designs are proposed: 1) we transform objective approximation into sequence-to-sequence modeling where MTMOO problem can be represented by tenxual tokenization. To operate under such auto-regressive modeling, we introduce a Large Language Model-based surrogate model that first encodes a MTMOO instance and then decodes objective values of unseen decision variables. To ensure stability in training the proposed model, we propose a two-stage offline training strategy that operates as a synergy of supervised tuning and RL fine-tuning, which first exploits offline dataset to fit existing knowledge and then leverages RL to enhance model's generalization performance. Extensive empirical results on the CEC2019 benchmark demonstrate that Q-MetaSur not only outperforms representative surrogate baselines in objective approximation accuracy, but also helps underlying evolutionary algorithms achieve both desired optimization convergence and improved pareto optimality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。