用扩散模型提建议，让自回归生成更精准。
Diffusion Is Your Friend in Show, Suggest and Tell
- 扩散模型不直接生成，而是给自回归过程提建议。
- 在COCO数据集上达到125.1的CIDEr-D，无需强化学习。
- 适合想提升文本生成质量的研究者和工程师。
扩散去噪模型在生成式计算机视觉任务中表现优异，但在离散领域仍难以超越标准自回归方法，最多仅能持平。本文提出新范式：利用扩散模型为自回归生成提供建议，而非替代。此举结合了扩散模型的双向建模与精炼能力，以及自回归模型的语言结构优势。我们构建了Show, Suggest and Tell（SST）模型，在类似设置下于COCO数据集上达到当前最优结果。SST在无需强化学习的情况下，取得125.1的CIDEr-D得分，分别领先自回归与扩散模型的最先进水平1.5和2.5点。通过大量实验验证，结果表明建议模块与生成质量呈正相关，揭示了一个尚未充分探索但极具潜力的方向。代码将开源于https://github.com/jchenghu/show_suggest_tell。
原文摘要 · Abstract (English)
Diffusion Denoising models demonstrated impressive results across generative Computer Vision tasks, but they still fail to outperform standard autoregressive solutions in the discrete domain, and only match them at best. In this work, we propose a different paradigm by adopting diffusion models to provide suggestions to the autoregressive generation rather than replacing them. By doing so, we combine the bidirectional and refining capabilities of the former with the strong linguistic structure provided by the latter. To showcase its effectiveness, we present Show, Suggest and Tell (SST), which achieves State-of-the-Art results on COCO, among models in a similar setting. In particular, SST achieves 125.1 CIDEr-D on the COCO dataset without Reinforcement Learning, outperforming both autoregressive and diffusion model State-of-the-Art results by 1.5 and 2.5 points. On top of the strong results, we performed extensive experiments to validate the proposal and analyze the impact of the suggestion module. Results demonstrate a positive correlation between suggestion and caption quality, overall indicating a currently underexplored but promising research direction. Code will be available at: https://github.com/jchenghu/show\_suggest\_tell.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。