arXiv:2512.11114cs.LGcs.AI2025-12被引 4

用Transformer实现无需重训练的多目标优化,一键生成优质设计方案。

In-Context Multi-Objective Optimization

  • 基于Transformer构建通用优化策略,全量上下文建模,无需任务定制
  • 单次前向传播完成设计推荐,提速50-1000倍,帕累托质量不降反升
  • 适合科学发现中频繁更换目标的高成本黑箱优化场景

在药物设计、自主系统等跨领域问题中,平衡多个竞争目标普遍存在。多目标贝叶斯优化通过概率代理模型和兼顾探索与利用的采集函数,为这类高成本黑箱问题提供解决方案。但实际应用中需针对每个任务定制代理模型与采集函数,难以迁移;且缺乏多步规划能力,反复拟合带来额外开销,尤其在并行或时间敏感的循环中。本文提出TAMO,一种完全可摊销、通用的多目标黑箱优化策略。TAMO采用Transformer架构,适应不同输入与目标维度,可在多样化数据集上预训练,并直接迁移到新任务而无需再训练:测试时仅需一次前向传播即可生成下一设计。通过强化学习预训练,最大化完整轨迹上的累积超体积改进,以整个查询历史为条件逼近帕累托前沿。在合成基准与真实任务中,TAMO实现快速提案,在严格评估预算下提案速度提升50-1000倍,同时保持或优于现有方法的帕累托质量。结果表明,Transformer可完全在上下文中执行多目标优化,消除任务专属代理拟合与采集函数工程,为科学发现工作流开辟了类基础模型、即插即用优化器的新路径。

原文摘要 · Abstract (English)

Balancing competing objectives is omnipresent across disciplines, from drug design to autonomous systems. Multi-objective Bayesian optimization is a promising solution for such expensive, black-box problems: it fits probabilistic surrogates and selects new designs via an acquisition function that balances exploration and exploitation. In practice, it requires tailored choices of surrogate and acquisition that rarely transfer to the next problem, is myopic when multi-step planning is often required, and adds refitting overhead, particularly in parallel or time-sensitive loops. We present TAMO, a fully amortized, universal policy for multi-objective black-box optimization. TAMO uses a transformer architecture that operates across varying input and objective dimensions, enabling pretraining on diverse corpora and transfer to new problems without retraining: at test time, the pretrained model proposes the next design with a single forward pass. We pretrain the policy with reinforcement learning to maximize cumulative hypervolume improvement over full trajectories, conditioning on the entire query history to approximate the Pareto frontier. Across synthetic benchmarks and real tasks, TAMO produces fast proposals, reducing proposal time by 50-1000x versus alternatives while matching or improving Pareto quality under tight evaluation budgets. These results show that transformers can perform multi-objective optimization entirely in-context, eliminating per-task surrogate fitting and acquisition engineering, and open a path to foundation-style, plug-and-play optimizers for scientific discovery workflows.

多目标优化Transformer黑箱优化科学发现

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。