提出新方法控制扩散语言模型的质量与多样性平衡
Understanding the Quality-Diversity Trade-off in Diffusion Language Models
- 用无分类器引导和随机钳制调节生成质量与多样性
- 在序列到序列任务中显著提升扩散语言模型性能
- 适合关注文本生成可控性的研究者与工程师
扩散模型在视觉、音频等连续数据领域取得巨大成功。尽管将扩散模型应用于离散文本数据面临挑战，近期工作通过在连续嵌入空间中建模实现了文本生成。然而，这些模型缺乏像自回归模型温度参数那样自然控制质量与多样性权衡的机制，限制了模型性能理解和生成质量。本文提出使用无分类器引导和随机钳制技术，在序列到序列任务中操控扩散语言模型的质量-多样性权衡，证明这些方法可有效提升模型表现。
原文摘要 · Abstract (English)
Diffusion models have seen immense success in modelling continuous data across a range of domains such as vision and audio. Despite the challenges of adapting diffusion models to discrete data, recent work explores their application to text generation by working in the continuous embedding space. However, these models lack a natural means to control the inherent trade-off between quality and diversity as afforded by the temperature hyperparameter in autoregressive models, hindering understanding of model performance and restricting generation quality. This work proposes the use of classifier-free guidance and stochastic clamping for manipulating the quality-diversity trade-off on sequence-to-sequence tasks, demonstrating that these techniques may be used to improve the performance of a diffusion language model.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。