arXiv:2410.11502cs.LGcs.AI2024-10ICLR被引 17

用排序学习替代传统回归,更精准找到最优设计。

Offline Model-Based Optimization by Learning to Rank

  • 不预测具体分数,而是学习设计间的相对优劣顺序。
  • 在多个任务上超越20种现有方法,显著提升优化效果。
  • 适合数据固定、无法新增实验的黑箱优化场景。

离线模型基于优化(Offline MBO)旨在仅使用预先收集的设计与得分数据集,寻找使黑箱函数最大化的设计方案。传统方法通常通过最小化均方误差(MSE)训练回归代理模型,并用优化器(如梯度上升)搜索最优解,但面临分布外误差风险:代理模型常高估得分,导致优化器陷入次优区域。尽管已有正则化与集成学习等改进方法,问题仍存在。本文指出,以MSE为损失的回归模型与离线MBO的核心目标——选择有潜力的设计——并不对齐。实验证明,最终设计方案质量与MSE相关性极弱,而与排序保持能力的相关性显著更强。为此,我们提出基于学习排序的代理模型,利用排序学习技术优先筛选相对表现更优的设计。理论上,排序损失的泛化误差可被有效控制。在多种任务上的实验表明,该方法性能优于20种现有方法。

原文摘要 · Abstract (English)

Offline model-based optimization (MBO) aims to identify a design that maximizes a black-box function using only a fixed, pre-collected dataset of designs and their corresponding scores. A common approach in offline MBO is to train a regression-based surrogate model by minimizing mean squared error (MSE) and then find the best design within this surrogate model by different optimizers (e.g., gradient ascent). However, a critical challenge is the risk of out-of-distribution errors, i.e., the surrogate model may typically overestimate the scores and mislead the optimizers into suboptimal regions. Prior works have attempted to address this issue in various ways, such as using regularization techniques and ensemble learning to enhance the robustness of the model, but it still remains. In this paper, we argue that regression models trained with MSE are not well-aligned with the primary goal of offline MBO, which is to select promising designs rather than to predict their scores precisely. Notably, if a surrogate model can maintain the order of candidate designs based on their relative score relationships, it can produce the best designs even without precise predictions. To validate it, we conduct experiments to compare the relationship between the quality of the final designs and MSE, finding that the correlation is really very weak. In contrast, a metric that measures order-maintaining quality shows a significantly stronger correlation. Based on this observation, we propose learning a ranking-based model that leverages learning to rank techniques to prioritize promising designs based on their relative scores. We show that the generalization error on ranking loss can be well bounded. Empirical results across diverse tasks demonstrate the superior performance of our proposed ranking-based models than twenty existing methods.

离线优化排序学习代理模型黑箱优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。