arXiv:2503.17286cs.LG2025-03中稿 · TMLR 2026综述被引 24

针对离线优化难题,提出系统性综述与方法分类。

Offline Model-Based Optimization: Comprehensive Review

  • 将离线优化分为代理建模与生成建模两大方向
  • 指出模型外推时存在认知不确定性导致性能虚假提升
  • 适合从事科学发现、材料设计等领域的研究人员参考

离线优化是科学与工程中的基础挑战,目标是在仅使用离线数据集的情况下优化黑箱函数。该场景在目标函数查询成本过高或不可行时尤为关键,应用涵盖蛋白质工程、材料发现、神经网络架构搜索等。主要难点在于准确估计数据之外的客观函数景观,而外推过程充满显著的认知不确定性。这种不确定性可能导致目标欺骗(奖励欺骗),利用模型在未见区域的不准确性,产生误导性的高性能估计。近年来,基于模型的优化(MBO)借助深度神经网络的泛化能力,发展出针对离线场景的代理与生成模型。通过精心设计的训练策略,这些模型对分布外问题更具鲁棒性,有助于发现更优设计。尽管该领域在加速科学发现方面影响日益扩大,但尚缺乏全面综述。为此,本文首次系统回顾离线MBO,形式化单目标与多目标设置,梳理最新基准与评估指标;将现有方法分为两类:代理建模(强调分布外区域的函数逼近精度)与生成建模(探索高维设计空间以发现高性能设计);最后分析核心挑战并提出未来发展方向,包括超智能系统安全控制。

原文摘要 · Abstract (English)

Offline optimization is a fundamental challenge in science and engineering, where the goal is to optimize black-box functions using only offline datasets. This setting is particularly relevant when querying the objective function is prohibitively expensive or infeasible, with applications spanning protein engineering, material discovery, neural architecture search, and beyond. The main difficulty lies in accurately estimating the objective landscape beyond the available data, where extrapolations are fraught with significant epistemic uncertainty. This uncertainty can lead to objective hacking(reward hacking), exploiting model inaccuracies in unseen regions, or other spurious optimizations that yield misleadingly high performance estimates outside the training distribution. Recent advances in model-based optimization(MBO) have harnessed the generalization capabilities of deep neural networks to develop offline-specific surrogate and generative models. Trained with carefully designed strategies, these models are more robust against out-of-distribution issues, facilitating the discovery of improved designs. Despite its growing impact in accelerating scientific discovery, the field lacks a comprehensive review. To bridge this gap, we present the first thorough review of offline MBO. We begin by formalizing the problem for both single-objective and multi-objective settings and by reviewing recent benchmarks and evaluation metrics. We then categorize existing approaches into two key areas: surrogate modeling, which emphasizes accurate function approximation in out-of-distribution regions, and generative modeling, which explores high-dimensional design spaces to identify high-performing designs. Finally, we examine the key challenges and propose promising directions for advancement in this rapidly evolving field including safe control of superintelligent systems.

离线优化模型基于优化科学发现生成模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。