用任意马尔可夫过程统一生成建模，支持新模型设计与多模态融合。
Generator Matching: Generative modeling with arbitrary Markov processes
- 通过条件生成器逼近全局生成器，实现通用生成建模。
- 在图像与多模态任务中验证，跳变过程超叠加提升性能。
- 兼容扩散、流匹配等方法，拓展生成模型设计空间。
我们提出生成器匹配（Generator Matching），一种基于任意马尔可夫过程的无模态生成建模框架。生成器表征马尔可夫过程的瞬时演化，我们借鉴流匹配思想：构造生成单个数据点的条件生成器，学习逼近生成完整数据分布的边际生成器。实验表明，生成器匹配统一了扩散模型、流匹配及离散扩散模型等方法，并拓展至跳变过程等未探索的马尔可夫过程。此外，该框架支持马尔可夫生成模型的叠加，可严谨构建多模态生成模型。我们在图像和多模态生成任务上进行了实证验证，结果表明与跳变过程的叠加能显著提升性能。
原文摘要 · Abstract (English)
We introduce Generator Matching, a modality-agnostic framework for generative modeling using arbitrary Markov processes. Generators characterize the infinitesimal evolution of a Markov process, which we leverage for generative modeling in a similar vein to flow matching: we construct conditional generators which generate single data points, then learn to approximate the marginal generator which generates the full data distribution. We show that Generator Matching unifies various generative modeling methods, including diffusion models, flow matching and discrete diffusion models. Furthermore, it expands the design space to new and unexplored Markov processes such as jump processes. Finally, Generator Matching enables the construction of superpositions of Markov generative models and enables the construction of multimodal models in a rigorous manner. We empirically validate our method on image and multimodal generation, e.g. showing that superposition with a jump process improves performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。