arXiv:2504.10283cs.LGstat.ML2025-04被引 18

提出统一框架α-Flow,让离散生成模型更高效准确。

$α$-Flow: A Unified Framework for Continuous-State Discrete Flow Matching Models

  • 基于信息几何构建α-表示的统一生成框架
  • 在图像、蛋白质序列生成中显著优于传统方法
  • 适合需要高精度离散生成的研究者使用

近期研究将流匹配框架拓展至离散生成建模。一类模型直接处理连续概率而非离散符号,我们称之为连续状态离散流匹配(CS-DFM)。现有模型在表示方式和几何假设上差异显著。本文提出统一框架,揭示不同模型本质是基于不同的α-表示。基于信息几何,引入α-Flow,其遵循统计流形的规范α-几何,在最小化广义动能方面具有最优性。理论证明,α-流匹配损失建立了离散负对数似然的统一变分界。在多个离散生成任务中全面评估α-流的不同实例,验证其在图像、蛋白质序列生成中的有效性,并首次探索了此前未研究的中间值几何结构。α-Flow在图像与蛋白质序列生成中显著优于其离散状态对应模型,且在语言建模中更好地捕捉熵结构。

原文摘要 · Abstract (English)

Recent efforts have extended the flow-matching framework to discrete generative modeling. One strand of models directly works with the continuous probabilities instead of discrete tokens, which we colloquially refer to as Continuous-State Discrete Flow Matching (CS-DFM). Existing CS-DFM models differ significantly in their representations and geometric assumptions. This work presents a unified framework for CS-DFM models, under which the existing variants can be understood as operating on different $α$-representations of probabilities. Building upon the theory of information geometry, we introduce $α$-Flow, a family of CS-DFM models that adheres to the canonical $α$-geometry of the statistical manifold, and demonstrate its optimality in minimizing the generalized kinetic energy. Theoretically, we show that the flow matching loss for $α$-flow establishes a unified variational bound for the discrete negative log-likelihood. We comprehensively evaluate different instantiations of $α$-flow on various discrete generation domains to demonstrate their effectiveness in discrete generative modeling, including intermediate values whose geometries have never been explored before. $α$-flow significantly outperforms its discrete-state counterpart in image and protein sequence generation and better captures the entropy in language modeling.

生成模型流匹配离散生成信息几何

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。