arXiv:2602.08012cs.LG2026-02被引 2

统一框架实现生成模型的可控融合与奖励引导合并。

A Unified Density Operator View of Flow Control and Merging

  • 提出概率空间统一框架,涵盖流控制与模型合并的极限情况。
  • 通过奖励引导融合,实现多预训练模型的最优组合,提升分子设计效率。
  • 适用于药物发现、分子生成等高维任务,支持安全约束与多样性控制。

大规模流模型与扩散模型的发展带来了两个核心算法挑战:(i) 预训练流的基于控制的奖励适应,(ii) 多模型集成,即流合并。现有方法分别处理这些问题,我们提出一个统一的概率空间框架,将两者作为极限情形包含在内,并支持奖励引导的流合并,实现多个预训练流的合理、任务感知组合(如在最大化药物发现效用的前提下合并先验)。该公式可表达生成模型密度上的一系列丰富算子,包括交集(如强制安全性)、并集(如组合多样性模型)、插值(如用于发现)、其奖励引导版本,以及通过生成电路实现的复杂逻辑表达式。随后,我们引入奖励引导流合并(RFM),一种镜面下降方案,将奖励引导流合并问题转化为一系列标准微调问题。我们首次为奖励引导和纯流合并提供了理论保证。最终,在示例场景中展示了该方法的能力,提供可视化解释,并应用于高维新分子设计和低能构象生成任务。

原文摘要 · Abstract (English)

Recent progress in large-scale flow and diffusion models raised two fundamental algorithmic challenges: (i) control-based reward adaptation of pre-trained flows, and (ii) integration of multiple models, i.e., flow merging. While current approaches address them separately, we introduce a unifying probability-space framework that subsumes both as limit cases, and enables reward-guided flow merging, allowing principled, task-aware combination of multiple pre-trained flows (e.g., merging priors while maximizing drug-discovery utilities). Our formulation renders possible to express a rich family of operators over generative models densities, including intersection (e.g., to enforce safety), union (e.g., to compose diverse models), interpolation (e.g., for discovery), their reward-guided counterparts, as well as complex logical expressions via generative circuits. Next, we introduce Reward-Guided Flow Merging (RFM), a mirror-descent scheme that reduces reward-guided flow merging to a sequence of standard fine-tuning problems. Then, we provide first-of-their-kind theoretical guarantees for reward-guided and pure flow merging via RFM. Ultimately, we showcase the capabilities of the proposed method on illustrative settings providing visually interpretable insights, and apply our method to high-dimensional de-novo molecular design and low-energy conformer generation.

生成模型流模型分子生成奖励引导

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。