用生成模型辅助低预算黑箱优化,不依赖奖励信号训练采样器。
Generative Refinement for Low-Budget Black-Box Optimization

- 将生成模型的先验与奖励信号解耦,使用已训练的采样器作为固定提案器。
- 在极低评估预算下仍能有效探索复杂几何结构与不可靠奖励的优化问题。
- 适合高成本、噪声大或失败率高的工程优化场景,如实验设计与超参调优。
黑箱优化无需梯度信息即可优化目标函数,但常需大量函数评估,在每次评估代价高昂时面临挑战。尤其当评估函数存在噪声或易失败,且最优解集中在搜索空间中狭窄、弯曲或不连通区域时更为严峻。现有基于生成模型的方法需大量评估以对齐采样器与奖励分布,难以适应低预算场景。本文提出SPARROW算法,完全解耦生成先验与奖励信号:可直接使用任何已训练于未评估数据的采样器,作为固定且结构化的提案算子;优化过程通过排名引导存档中已评估候选者的演化实现。该方法能有效导航复杂几何结构,容忍不可靠奖励信号,并在极低评估预算下表现优异。我们提供了关于采样器支持集的渐近收敛性证明,并在具有不可靠奖励和几何复杂性的任务上验证了其强大性能。
原文摘要 · Abstract (English)
Black-box optimization is a fundamental science and engineering tool that makes it possible to optimize objectives without gradient information. Unfortunately, as it often requires many function evaluations, it can be challenging when each one is costly. This is especially true when the evaluation function is noisy or failure-prone, and when high-performing solutions are confined to thin, curved, or disconnected regions of the search space. Existing methods leveraging generative models to navigate these subspaces are built to sample from reward-aligned distributions. As a result, they require a large number of evaluations to align their sampler effectively, making them impractical in low-budget settings. We propose SPARROW, an algorithm that completely decouples the generative prior from the reward signal. SPARROW can use any sampler with a known corruption process and trained on unevaluated data, as a fixed, structured proposal operator. Optimization proceeds by rank-based guidance over an archive of evaluated candidates. SPARROW can navigate complex geometries, handle unreliable reward signals, and perform effective optimization under very low evaluation budgets. We provide asymptotic convergence guarantees over the sampler support and demonstrate strong empirical performance on problems with unreliable rewards and geometrically complex landscapes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。