用合成数据训练基础模型,实现高效多目标优化
FoMEMO: Towards Foundation Models for Expensive Multi-objective Optimization
- 基于合成数据预训练基础模型,支持快速上下文优化
- 在未知问题上达到优越泛化性能,无需后续训练
- 适合需要快速响应的复杂优化场景
昂贵的多目标优化在众多现实场景中普遍存在,由于评估次数受限,样本效率至关重要。现有方法或需为每个新问题重新构建高斯过程代理模型,或依赖大量真实域实验进行深度学习模型预训练,难以泛化且不适用于新兴应用。为此,我们提出一种新范式FoMEMO(面向昂贵多目标优化的基础模型),可基于任意领域轨迹和用户偏好建立基础模型,并通过预测的偏好聚合后验实现快速上下文优化。我们证明,仅用数亿条合成数据预训练即可在未知问题上实现优异泛化与优化性能,且后续优化过程无需任何模型训练或更新。
原文摘要 · Abstract (English)
Expensive multi-objective optimization is a prevalent and crucial concern in many real-world scenarios, where sample-efficiency is vital due to the limited evaluations to recover the true Pareto front for decision making. Existing works either involve rebuilding Gaussian process surrogates from scratch for each objective in each new problem encountered, or rely on extensive past domain experiments for pre-training deep learning models, making them hard to generalize and impractical to cope with various emerging applications in the real world. To address this issue, we propose a new paradigm named FoMEMO (Foundation Models for Expensive Multi-objective Optimization), which enables the establishment of a foundation model conditioned on any domain trajectory and user preference, and facilitates fast in-context optimization based on the predicted preference-wise aggregated posteriors. Rather than accessing extensive real-world domain experiments for training, we demonstrate that pre-training the foundation model with a diverse set of hundreds of millions of synthetic data can lead to superior generalization and optimization performance to unknown problems, without necessitating any subsequent model training or updates in the following optimization process.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。