用Mamba架构实现低计算量高质量人脸语义合成
SISMA: Semantic Face Image Synthesis with Mamba
- 采用Mamba替代扩散模型中的注意力层，降低计算复杂度
- 在CelebAMask-HQ上实现更优的FID分数且推理速度提升3倍
- 适合追求高效生成、资源受限场景的应用者
扩散模型在人脸语义图像合成（SIS）中广受欢迎，但其训练和推理计算开销大，源于注意力层的二次复杂度。本文提出基于Mamba的新架构SISMA，通过语义掩码控制形状生成高质量图像，显著降低计算需求。在CelebAMask-HQ上的全面实验表明，该架构不仅取得更优的FID分数，且运行速度达当前最先进方法的三倍，证明其是变压器模型的轻量级可行替代方案。
原文摘要 · Abstract (English)
Diffusion Models have become very popular for Semantic Image Synthesis (SIS) of human faces. Nevertheless, their training and inference is computationally expensive and their computational requirements are high due to the quadratic complexity of attention layers. In this paper, we propose a novel architecture called SISMA, based on the recently proposed Mamba. SISMA generates high quality samples by controlling their shape using a semantic mask at a reduced computational demand. We validated our approach through comprehensive experiments with CelebAMask-HQ, revealing that our architecture not only achieves a better FID score yet also operates at three times the speed of state-of-the-art architectures. This indicates that the proposed design is a viable, lightweight substitute to transformer-based models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。