让生成模型主动发现新分子,突破数据分布限制。
Active Flow Expansion for Out-of-Distribution Discovery: from Theory to Molecules

- 通过验证器反馈迭代生成新数据,扩展模型可生成区域。
- 在多种分子任务中,有效覆盖远超初始模型的合法设计空间。
- 适合需要探索未知分子结构的药物与蛋白设计研究者。
标准流模型和扩散模型预训练仅匹配已有数据分布(如分子),通常只覆盖有效设计空间的一小部分。但在生成式发现中,目标是采样前所未有的合法新设计,这些设计在现有模型下概率极低且不可达。为此,我们摒弃数据分布匹配思路,转而关注生成集合:模型赋予非零概率的区域。由此提出一种新的分布外流模型学习原则——扩大生成集合以提升对合法设计空间的覆盖率。我们提出主动流扩展(ActFlow),一种持续预训练方法,利用验证器反馈,通过在学习到的流表示空间中主动探索生成合成数据,并迭代适应,逐步拓展模型至新的合法区域。理论上,我们首次建立分布外流建模的统计学习保证,将生成集扩张视为学习表示上的局部到全局可达性过程。实验上,在小分子、中等大小类药分子、治疗肽及蛋白序列设计任务中,采用合适的分布外生成建模指标评估,结果表明,ActFlow显著扩展了合法覆盖范围,远超初始预训练模型,且大幅优于广泛使用的合成流预训练方法。
原文摘要 · Abstract (English)
Standard flow and diffusion pre-training matches the distribution of available data (e.g., molecules), which often covers only a small fraction of the valid design space. In generative discovery, however, one aims to sample valid new-to-nature designs, assigned negligible probability under, and thus inaccessible to, standard models fitted to the observed data. To overcome this limitation, we depart from data distribution matching and view a generative model through its generable set: the region it covers with non-negligible probability. This allows to introduce a new learning principle for out-of-distribution flow modeling: enlarging a model's generable set to increase coverage of the valid design space. We propose Active Flow Expansion (ActFlow), a continued pre-training method that employs verifier feedback to expand a pre-trained model over new valid regions by iteratively adapting to synthetic data generated through active exploration in the learned flow representation. Theoretically, we establish to our knowledge first-of-their-kind statistical learning guarantees for out-of-distribution flow modeling, analyzing generable set expansion as a local-to-global reachability process over a learned representation. Empirically, we assess ActFlow with suitable out-of-distribution generative modeling metrics across small organic molecules, mid-sized drug-like molecules, therapeutic peptides, and protein sequence design tasks. Results show that ActFlow expands valid coverage far beyond the region modeled by the initial pre-trained model, significantly outperforming widely adopted synthetic flow pre-training methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。