用结构化Transformer学习目标函数结构，提升设计优化效果
Cliqueformer: Model-Based Optimization with Structured Transformers
- 基于功能图模型构建结构化Transformer，捕捉目标函数内在关系
- 在化学与基因设计任务中表现优于现有方法，解决分布偏移问题
- 适合需要高效探索复杂设计空间的研究者
大型神经网络在预测任务上表现优异，但应用于蛋白质工程或材料发现等设计问题时，需解决离线模型基础优化（MBO）问题。尽管预测模型未必直接转化为有效设计，近期的MBO算法已融合强化学习与生成建模方法。同时，理论研究表明，利用目标函数的结构可提升MBO性能。我们提出Cliqueformer，一种基于Transformer的架构，通过功能图模型（FGM）学习黑箱函数的结构，无需依赖显式的保守策略即可应对分布偏移。在化学与遗传设计等多种任务中，Cliqueformer均展现出优于现有方法的性能。
原文摘要 · Abstract (English)
Large neural networks excel at prediction tasks, but their application to design problems, such as protein engineering or materials discovery, requires solving offline model-based optimization (MBO) problems. While predictive models may not directly translate to effective design, recent MBO algorithms incorporate reinforcement learning and generative modeling approaches. Meanwhile, theoretical work suggests that exploiting the target function's structure can enhance MBO performance. We present Cliqueformer, a transformer-based architecture that learns the black-box function's structure through functional graphical models (FGM), addressing distribution shift without relying on explicit conservative approaches. Across various domains, including chemical and genetic design tasks, Cliqueformer demonstrates superior performance compared to existing methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。