提出可动态增删元素的生成模型,解决序列长度未知时的建模难题。
Branching Flows: Discrete, Continuous, and Manifold Flow Matching with Splits and Deletions
- 用可分裂与消亡的二叉树结构模拟元素演化过程。
- 在分子、抗体和蛋白质生成任务中实现稳定训练与高质量生成。
- 支持离散、连续及混合空间建模,适用于多种生成场景。
扩散与流匹配方法在图像生成或蛋白质折叠设计等连续状态空间中表现良好,也适用于元素数量固定的离散场景(如大语言模型)。但当生成序列长度未知时(如响应长度或蛋白链氨基酸数),传统方法需额外处理。本文提出分支流(Branching Flows)框架,通过学习元素在二叉树森林中随机分裂与消亡的速率,动态控制生成过程中元素数量。该方法可与任意离散集、连续欧氏空间、光滑流形及混合型乘积空间上的流匹配过程结合。在小分子生成(多模态)、抗体序列生成(离散)和蛋白质骨架生成(多模态)三类任务中验证了其有效性,证明其具备稳定的训练目标与强大的分布拟合能力,并拓展了生成模型的新功能。
原文摘要 · Abstract (English)
Diffusion and flow matching approaches to generative modeling have shown promise in domains where the state space is continuous, such as image generation or protein folding & design, and discrete, exemplified by diffusion large language models. They offer a natural fit when the number of elements in a state is fixed in advance (e.g. images), but require ad hoc solutions when, for example, the length of a response from a large language model, or the number of amino acids in a protein chain is not known a priori. Here we propose Branching Flows, a generative modeling framework that, like diffusion and flow matching approaches, transports a simple distribution to the data distribution. But in Branching Flows, the elements in the state evolve over a forest of binary trees, branching and dying stochastically with rates that are learned by the model. This allows the model to control, during generation, the number of elements in the sequence. We also show that Branching Flows can compose with any flow matching base process on discrete sets, continuous Euclidean spaces, smooth manifolds, and `multimodal' product spaces that mix these components. We demonstrate this in three domains: small molecule generation (multimodal), antibody sequence generation (discrete), and protein backbone generation (multimodal), and show that Branching Flows is a capable distribution learner with a stable learning objective, and that it enables new capabilities.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。