用信息增益优化掩码扩散模型采样，提升生成质量
Improving Sampling for Masked Diffusion Models via Information Gain
- 基于双向结构评估每个词的长期信息价值，避免局部贪心
- 推理准确率提升2.9%-11.6%，创意写作胜率62.8%
- 无需训练，适配多种生成任务
掩码扩散模型（MDMs）支持灵活的解码顺序，但现有采样器仍以局部确定性为主，忽略其对后续生成的影响。我们发现这种短视会增加累积不确定性，导致生成效果不佳。为此，提出**信息增益采样器**（Info-Gain Sampler），一种无需训练的解码方法，利用MDMs的双向结构，在即时不确定性与剩余掩码位置的信息增益之间取得平衡。在推理、编码、创意写作和图像生成等任务中，该方法持续优于现有MDM采样器，平均推理准确率提升2.9–11.6个百分点，创意写作任务平均胜率达62.8%。代码已开源：https://github.com/yks23/Information-Gain-Sampler。
原文摘要 · Abstract (English)
Masked Diffusion Models (MDMs) enable flexible decoding orders, yet existing samplers remain largely greedy, selecting locally certain tokens without accounting for their downstream effects. We show that this myopia can increase cumulative uncertainty and lead to suboptimal generation. To address this, we propose the **Info-Gain Sampler**, a training-free decoding method that uses the bidirectional structure of MDMs to balance immediate uncertainty with the information gained over remaining masked positions. Across reasoning, coding, creative writing, and image generation tasks, Info-Gain Sampler consistently outperforms existing MDM samplers, improving average reasoning accuracy by 2.9--11.6 percentage points and achieving a 62.8% average win rate in creative writing. The code is available at https://github.com/yks23/Information-Gain-Sampler.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。