提出Simplax增强离散扩散模型，提升生成质量与解题准确率。
Simplex Relaxation for Discrete Diffusion
- 引入辅助单纯形变量，保持原腐蚀过程不变
- 在OpenWebText上改善困惑度-熵权衡，Sudoku任务达最优解题率
- 适合需高精度生成与推理的离散数据建模场景
针对类别型数据生成的离散扩散模型，其定义依赖于腐蚀核，决定中间状态空间与反向预测问题。本文研究均匀离散扩散，探讨是否可在不改变原始类别腐蚀过程的前提下丰富训练目标与反向转移。提出Simplax，一种精确的狄利克雷-类别增强方法，将每个被腐蚀的类别状态与一个辅助单纯形变量耦合，同时保持原始均匀扩散过程作为其类别边缘分布。该增强方法导出可计算的Rao-Blackwell化反向桥目标与对应的随机反向采样器，且保留被腐蚀类别状态作为去噪器输入。实验表明，Simplax在无条件OpenWebText生成中提升了生成困惑度-熵权衡；在数独任务中，仅用30个线索训练的模型，在所有线索密度下均优于对比方法，包括最小唯一可解的17线索情形，并在无条件生成中达到最高有效性。
原文摘要 · Abstract (English)
Discrete diffusion models for categorical generation are defined by a corruption kernel, which determines the intermediate state space and the associated reverse prediction problem. We study uniform discrete diffusion and ask whether its training objective and reverse transitions can be enriched without changing the underlying categorical corruption process. We introduce Simplax, an exact Dirichlet--categorical augmentation that couples each corrupted categorical state with an auxiliary simplex-valued variable while preserving the original uniform diffusion process as its categorical marginal. This augmentation yields a tractable Rao--Blackwellized reverse-bridge objective and a corresponding stochastic reverse sampler, while retaining the corrupted categorical state as the denoiser input. Empirically, Simplax improves the generative perplexity--entropy tradeoff on unconditional OpenWebText generation. On Sudoku, a model trained exclusively on $30$-clue puzzles achieves the highest accuracy among the compared methods across all evaluated clue densities, including the minimum uniquely solvable $17$-clue regime, and also achieves the highest validity in unconditional generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。