将掩码图像建模与归一化流结合,实现端到端图像生成。
MIMFlow: Integrating Masked Image Modeling with Normalizing Flows for End-to-End Image Generation

- 用变分自编码器从掩码图像中提取语义特征,解耦生成任务
- 在ImageNet上达FID 2.50,比同类模型提升32.8%
- 仅用128个令牌,性能超越标准模型
归一化流(NFs)虽能精确密度估计和采样,但其严格可逆性常导致模型过度关注低层像素细节,难以捕捉高层语义结构。掩码图像建模(MIM)虽在表征学习中表现优异,但其与生成流程的集成仍多为模块化、分离式设计。本文提出MIMFlow,一种统一的端到端框架,联合优化潜在语义、像素重建与生成流。通过使用变分自编码器(VAE)编码器从掩码图像中推断语义潜变量,MIMFlow实现了生成任务的原理性解耦:归一化流专注于建模简化后的低频语义流形,而专用解码器负责高频合成。该设计有效缓解了归一化流的固有容量瓶颈,使模型更注重全局结构一致性而非冗余噪声。在ImageNet 256×256上的实验表明,MIMFlow-L达到71.3%线性探测准确率和FID 2.50。尽管仅使用128个令牌(比标准模型少50%),其性能仍较同类规模归一化流基线提升32.8%。代码已开源。
原文摘要 · Abstract (English)
Normalizing Flows (NFs) are powerful generative models capable of exact density estimation and sampling. However, their strict invertibility often forces the model to exhaust its capacity on low-level pixel details, hindering the capture of high-level semantic structures. While Masked Image Modeling (MIM) has excelled in representation learning, its integration into generative pipelines has remained largely modular and disjointed. In this paper, we propose MIMFlow, a unified end-to-end framework that jointly optimizes latent semantics, pixel reconstruction, and generative flow. By employing a VAE encoder to infer semantic latent from masked images, MIMFlow achieves a principled decoupling of the generative task: the Normalizing Flow focuses on modeling a simplified, low-frequency semantic manifold, while a specialized decoder handles high-frequency synthesis. This design effectively resolves the inherent capacity bottleneck of NFs, allowing the model to prioritize global structural coherence over redundant noise. Empirical results on ImageNet 256$\times$256 show that MIMFlow-L reaches 71.3\% linear probing accuracy and an FID of 2.50. Despite using only 128 tokens (50\% fewer than standard models), it yields a 32.8\% performance gain over similar-scale NF baselines. Our code is available at https://github.com/MCG-NJU/MIMFlow.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。