用[MASK]统一生成与扩散模型，实现高效多任务建模
[MASK] is All You Need
- 以离散状态模型为桥梁，统一掩码生成与非自回归扩散模型
- 在ImageNet256等数据集上达到或超越现有离散模型性能
- 仅需一次训练即可支持灵活条件采样，适合多任务应用
在生成模型中，基于下一集合预测的掩码生成模型与基于下一噪声预测的非自回归模型（如扩散模型）备受关注。本文提出使用离散状态模型连接两者，并探索其在视觉领域的可扩展性。首先，在统一设计空间中系统分析时间步独立性、噪声调度、温度、引导强度等因子的可扩展性。其次，将典型判别任务（如图像分割）重构为离散状态模型中对[MASK]标记的去掩码过程，从而支持多种采样策略，仅需一次训练即可建模联合分布并实现灵活条件采样。上述探索形成名为离散插值器（Discrete Interpolants）的框架，在ImageNet256、MS COCO和视频数据集FaceForensics上均取得当前最优或具有竞争力的性能。结果表明，通过在离散状态模型中引入[MASK]，可融合掩码生成与非自回归扩散模型，并贯通生成与判别任务。
原文摘要 · Abstract (English)
In generative models, two paradigms have gained attraction in various applications: next-set prediction-based Masked Generative Models and next-noise prediction-based Non-Autoregressive Models, e.g., Diffusion Models. In this work, we propose using discrete-state models to connect them and explore their scalability in the vision domain. First, we conduct a step-by-step analysis in a unified design space across two types of models including timestep-independence, noise schedule, temperature, guidance strength, etc in a scalable manner. Second, we re-cast typical discriminative tasks, e.g., image segmentation, as an unmasking process from [MASK] tokens on a discrete-state model. This enables us to perform various sampling processes, including flexible conditional sampling by only training once to model the joint distribution. All aforementioned explorations lead to our framework named Discrete Interpolants, which enables us to achieve state-of-the-art or competitive performance compared to previous discrete-state based methods in various benchmarks, like ImageNet256, MS COCO, and video dataset FaceForensics. In summary, by leveraging [MASK] in discrete-state models, we can bridge Masked Generative and Non-autoregressive Diffusion models, as well as generative and discriminative tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。