通过分段扩散提升长文本生成的连贯性与可控性
Segment-Level Diffusion: A Framework for Controllable Long-Form Generation with Diffusion Language Models
- 将长文本分段处理，分块建模潜在表示
- 在四个数据集上优于基线模型的流畅度与一致性
- 适合需要高质量长文本生成的研究者使用
扩散模型在文本生成中展现出潜力，但难以生成长篇、连贯且上下文准确的文本。词级扩散未显式建模词序依赖，且仅作用于短固定窗口；段落级扩散则难以学习长文本的鲁棒表示。为此，我们提出分段级扩散（Segment-Level Diffusion, SLD）框架，通过文本分段、对抗与对比学习的鲁棒表示训练，以及改进的隐空间引导机制，提升扩散文本生成能力。将长文本分段为多个潜在表示，再由自回归解码器逐段生成，简化扩散预测并增强可扩展性。在四个数据集上的实验表明，相较于其他扩散与自回归基线模型，SLD 在自动评估和人工评估中均实现了具有竞争力或更优的流畅性、连贯性与上下文兼容性。
原文摘要 · Abstract (English)
Diffusion models have shown promise in text generation, but often struggle with generating long, coherent, and contextually accurate text. Token-level diffusion doesn't model word-order dependencies explicitly and operates on short, fixed output windows, while passage-level diffusion struggles with learning robust representations for long-form text. To address these challenges, we propose Segment-Level Diffusion (SLD), a framework that enhances diffusion-based text generation through text segmentation, robust representation training with adversarial and contrastive learning, and improved latent-space guidance. By segmenting long-form outputs into multiple latent representations and decoding them with an autoregressive decoder, SLD simplifies diffusion predictions and improves scalability. Experiments on four datasets demonstrate that, when compared to other diffusion and autoregressive baselines SLD achieves competitive or superior fluency, coherence, and contextual compatibility in automatic and human evaluations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。