无需训练即可修正生成错误，提升多模态扩散模型质量
Training-Free Self-Correction for Multimodal Masked Diffusion Models
- 利用预训练模型的归纳偏置实现无训练自纠错
- 减少采样步数的同时显著提升图文生成与多模态理解性能
- 兼容多种掩码扩散架构，适合快速部署的生成任务
掩码扩散模型已成为文本与多模态生成的强大框架。然而，其采样过程同时更新多个标记且视生成结果为不可修改，导致早期错误无法修正，引发误差累积。本文重新审视现有自纠错方法，发现其受限于额外训练需求或依赖错位的概率估计。为此，提出一种无需训练的自纠错框架，利用预训练掩码扩散模型的归纳偏置。不修改模型参数，也不引入辅助评估器，该方法在文本到图像生成和多模态理解任务上显著提升生成质量，同时减少采样步骤。该框架可泛化至不同掩码扩散架构，体现其鲁棒性与实用性。代码见 https://github.com/huge123/FreeCorrection。
原文摘要 · Abstract (English)
Masked diffusion models have emerged as a powerful framework for text and multimodal generation. However, their sampling procedure updates multiple tokens simultaneously and treats generated tokens as immutable, which may lead to error accumulation when early mistakes cannot be revised. In this work, we revisit existing self-correction methods and identify limitations stemming from additional training requirements or reliance on misaligned likelihood estimates. We propose a training-free self-correction framework that exploits the inductive biases of pre-trained masked diffusion models. Without modifying model parameters or introducing auxiliary evaluators, our method significantly improves generation quality on text-to-image generation and multimodal understanding tasks with reduced sampling steps. Moreover, the proposed framework generalizes across different masked diffusion architectures, highlighting its robustness and practical applicability. Code can be found in https://github.com/huge123/FreeCorrection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。