提出混合扩散模型CANDI，让离散数据也能用连续扩散的优势。
CANDI: Hybrid Discrete-Continuous Diffusion Models
- 分离离散与连续噪声污染机制，实现双路径学习。
- 文本生成中低计算量下超越掩码扩散模型性能。
- 适合想用现成分类器指导离散生成的研究者。
尽管连续扩散在图像生成等连续领域表现优异，但其直接应用于离散数据时效果不佳。为理解这一差距，我们引入「词元可辨识性」分析框架，揭示高斯噪声通过两种机制破坏离散数据：词元身份丢失与连续秩退化。这两种机制随词表规模变化速率不同，导致时间上的不匹配，迫使在学习连续几何与离散结构间做出权衡。为此，我们提出CANDI（Continuous ANd DIscrete diffusion）——一种解耦离散与连续污染的混合框架，实现两者的并行学习。该方法使连续扩散在离散空间中获益：在可控生成中，可通过简单梯度叠加使用现成分类器进行引导；在文本生成中，低NFE（Number of Function Evaluations）下优于掩码扩散模型，证明了学习连续梯度对离散空间的价值。代码已公开于项目页面：https://patrickpynadath1.github.io/candi-lander。
原文摘要 · Abstract (English)
While continuous diffusion has shown remarkable success in continuous domains such as image generation, its direct application to discrete data has underperformed pure discrete formulations. To understand this gap, we introduce token identifiability, an analytical framework characterizing how Gaussian noise corrupts discrete data through two mechanisms: discrete identity corruption and continuous rank degradation. We reveal that these mechanisms scale differently with vocabulary size, creating a temporal dissonance that forces a tradeoff between learning continuous geometry and discrete structure. To address this, we propose CANDI (Continuous ANd DIscrete diffusion), a hybrid framework that decouples discrete and continuous corruption, enabling simultaneous learning of both. This unlocks the benefits of continuous diffusion for discrete spaces: on controlled generation, CANDI enables classifier-based guidance with off-the-shelf classifiers through simple gradient addition; on text generation, CANDI outperforms masked diffusion at low NFE, demonstrating the value of learning continuous gradients for discrete spaces. We include the code on the project page: https://patrickpynadath1.github.io/candi-lander.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。