一次性生成离散数据的新模型，效率远超传统方法。
Coupling Models for One-Step Discrete Generation
- 直接学习离散序列与高斯隐变量的耦合关系，一步生成
- 文本、基因增强子、二值图像生成任务上性能提升33%~46%
- 无需复杂流模型或人工设计噪声耦合，适合快速生成场景
离散结构的生成建模在深度学习中应用广泛，涵盖生物序列设计、代码生成和大语言模型等。然而，现有方法多依赖自回归解码或迭代优化，生成过程缓慢。本文提出耦合模型（Coupling Models），一种一次性生成离散数据的新方法，通过学习离散序列与高斯隐变量之间的直接耦合关系，构建专用解码器实现单步生成。该模型避免了复杂的单纯形空间连续流和人为设定的数据-噪声耦合方式。实验表明，耦合模型在各领域均优于最强的一步生成基线：在LM1B文本生成中将困惑度降低33%，在Fly Brain enhancer设计任务中提升18%，在MNIST-Binary图像生成中FID降低46%。结果表明，有效的一次性离散生成高度依赖于生成前数据与噪声的耦合方式。代码已开源。
原文摘要 · Abstract (English)
Generative modeling over discrete structures underpins applications across deep learning, from biological sequence design and code generation to large language models, yet generation often remains sequential, relying on autoregressive decoding or iterative refinement. In this work, we introduce Coupling Models(Coupling Models), a one-step discrete generative model that learns a direct coupling between discrete sequences and Gaussian latents. Unlike recent distillation methods that compress a pretrained multi-step sampler into a few steps, Coupling Model trains a purpose-built decoder to invert this coupling and generate samples in a single step. The model also avoids complex continuous flows over the simplex and hand-specified data-to-noise couplings. Empirically,Coupling Model improves the strongest one-step baselines in each domain: it reduces LM1B text-generation perplexity by 33% at its lowest-perplexity operating point, Fly Brain enhancer-design FBD by 18%, and MNIST-Binary FID by 46%. These results suggest that effective one-step discrete generation depends strongly on how data and noise are coupled before decoding. Code is available at https://github.com/pengzhangzhi/Coupling-Models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。