用双向语言建模提升文本生成图像的语义空间精度
BLM-SGAN: Bidirectional Language Modeling for Semantic-Spatial Text-to-Image Generation
- 引入BERT注意力机制捕捉长程上下文依赖
- 生成鸟类图像的Inception Score达5.45，优于多个基线模型
- 适合需要精准语义对齐的图像生成研究者
尽管文本到图像（T2I）生成已取得显著进展，但在自然语言处理与计算机视觉交叉领域仍面临挑战。基于生成对抗网络（GAN）的T2I模型虽有改进，但仍存在难以捕捉长距离依赖、梯度消失及顺序处理局限等问题。为此，本文提出BLM-SGAN，一种融合双向语言建模的语义-空间文本到图像生成模型。该模型利用BERT的注意力机制，有效捕获丰富上下文信息并高效处理长序列。实验表明，BLM-SGAN在生成高质量鸟类图像方面表现优异，获得5.45 ± 0.08的Inception Score，超越SSA-GAN、DF-GAN、SD-GAN和AttnGAN等多个先进模型。代码已开源：https://github.com/haidy-maher/BLM-SGAN-Text-to-Image-Generation。
原文摘要 · Abstract (English)
Despite the success of image generation from text descriptions, it still faces challenges that are difficult to overcome in domains such as natural language processing (NLP) and computer vision (CV). Recent advancements in text-to-image (T2I) models, particularly those utilizing generative adversarial networks (GANs), have significantly improved the synthesis of realistic images across various domains. However, existing GAN-based T2I models still encounter key challenges, such as difficulty in capturing long-range dependencies, vanishing gradients, and the limitations of sequential processing. To address these issues, we introduce BLM-SGAN, a novel model that incorporates Bidirectional Language Modeling for Semantic-Spatial Text-to-Image Generation. BLM-SGAN leverages BERT's attention mechanisms to capture rich contextual information and efficiently manage extended sequences. Our model demonstrates state-of-the-art performance, with an Inception Score (IS) of 5.45 +/- 0.08, surpassing several competitive models such as SSA-GAN, DF-GAN, SD-GAN, and AttnGAN. BLM-SGAN effectively generates highly realistic images of birds from detailed text descriptions. The implementation code is available at: https://github.com/haidy-maher/BLM-SGAN-Text-to-Image-Generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。