让生成顺序随数据自适应，提升分子生成质量
Insertion Based Sequence Generation with Learnable Order Dynamics
- 用可学习的动态顺序替代固定生成流程
- 分子生成任务中质量提升最高达17.5%
- 适合需要高质量结构化序列生成的研究者
现有基于插入的掩码扩散模型通过交替插入标记与解掩码生成序列，但使用固定生成时序，与数据无关。对于图和分子等结构化序列，学习数据相关的生成顺序可降低动作空间的不确定性，提升生成质量。本文提出LoFlexMDM，一种具有可学习生成顺序的插入式掩码扩散模型，能自动学习数据依赖的插入与解掩码速率。我们拓展离散流匹配框架以支持变长序列，提出可计算的时序参数化方法及联合训练目标。在全新分子和片段约束分子生成任务中，相较于FlexMDM，LoFlexMDM分别实现最高17.5%和6.7%的质量提升。结果表明，学习目标生成顺序可在保持训练可扩展性的同时改进插入式扩散模型。代码已开源：https://github.com/dhruvdcoder/LoFlexMDM。
原文摘要 · Abstract (English)
Existing insertion-based masked diffusion models that generate sequences by interleaving token insertion with unmasking use fixed schedules that are not dependent on the data. For structured sequences like graphs and molecules, learning data-dependent generation orders can improve generation quality by reducing uncertainty over the action space. We propose LoFlexMDM, an insertion-based masked diffusion model with learnable order dynamics that learns data-dependent insertion and unmasking rates. We generalize the discrete flow matching framework to work with variable-length sequences, propose a tractable schedule parameterization and a training objective for joint training of the generator and the target order dynamics. On De Novo and fragment-constrained molecule generation, LoFlexMDM improves sample quality over FlexMDM by up to 17.5% and 6.7%, respectively. These results show that learning the target generation order can improve insertion-based diffusion models without giving up tractable training. We open source the code at https://github.com/dhruvdcoder/LoFlexMDM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。