arXiv:2409.13739cs.CL2024-09被引 6

用扩散模型生成表格文本,质量与多样性更均衡。

Table-to-Text Generation with Pretrained Diffusion Models

  • 将扩散模型适配表格转文本任务,探索采样与聚合策略
  • 在相同条件下,扩散模型比自回归模型更平衡质量与多样性
  • 推荐用MBR聚合多样本提升质量,或用DPM-Solver++加速生成

扩散模型在多种文本生成任务中展现出卓越性能。本文系统研究其在表格转文本任务中的应用,通过适配扩散模型并深入分析训练多个方面。实验涵盖采样策略影响,引入最新加速器DPM-Solver++;测试了ROVER与最小贝叶斯风险(MBR)等预测聚合方法;考察了预训练阶段及生成长度约束的影响。同时对比了扩散模型与自回归文本生成模型在不同温度设置下的多样性表现。关键发现:扩散模型在质量与多样性之间取得更好平衡,而自回归模型难以兼顾两者。进一步研究表明,为获得最高质量,建议使用严格长度约束的常规采样生成多份输出,再通过MBR聚合;若追求速度并接受较低多样性,则可采用DPM-Solver++加速器。结果表明,扩散模型在表格转文本任务中表现相当优异,具备成为该领域有前景的研究方向。

原文摘要 · Abstract (English)

Diffusion models have demonstrated significant potential in achieving state-of-the-art performance across various text generation tasks. In this systematic study, we investigate their application to the table-to-text problem by adapting the diffusion model to the task and conducting an in-depth analysis. Our experiments cover multiple aspects of diffusion models training. We explore sampling strategy influence by inducing recent diffusion model accelerator DPM-Solver++ into our core model. We have tested different prediction aggregation methods, like ROVER and Minimum Bayes-Risk (MBR). Our studies cover the impact of the pre-training phase in diffusion models and the generation length constraints influence. We also have compared diffusion model generation with auto-regressive text-to-text models with different temperature settings for diversity evaluation. Our key observation is that diffusion models demonstrate the balance between quality and diversity while auto-regressive text-to-text models are not successful at handling both at the same time. Furthermore, we found out that to achieve the highest quality possible, it is preferable to use a regular sampler with the strictest length constraint to create multiple samples, and then use MBR to aggregate the predictions. However, if you are prepared to give up high level of diversity and to accelerate the process, you can also utilize a fast sampler DPM-Solver++. Our findings reveal that diffusion models achieve comparable results in the table-to-text domain, highlighting their viability in the table-to-text challenge as a promising research direction.

表格生成扩散模型文本生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。