用可逆模型实现图文互生，一训练两用。
Image Generation from Image Captioning -- Invertible Approach
- 构建可逆神经网络，打通图文嵌入空间的双向映射
- 仅训练图像描述任务，即可通过反演生成新图像
- 无需额外训练，适合跨模态生成与迁移应用
本研究旨在构建一个能同时完成图像描述与图像生成双重任务的模型，且仅需在单一任务上进行训练。核心思想是训练一个可逆模型，学习图像与文本嵌入之间的双射映射关系。一旦该可逆模型在图像描述任务上高效训练完成，便可通过对文本嵌入进行反演，生成对应的新图像，而无需任何额外训练。本文提出一种简洁的可逆神经网络架构，并报告了当前的研究成果。
原文摘要 · Abstract (English)
Our work aims to build a model that performs dual tasks of image captioning and image generation while being trained on only one task. The central idea is to train an invertible model that learns a one-to-one mapping between the image and text embeddings. Once the invertible model is efficiently trained on one task, the image captioning, the same model can generate new images for a given text through the inversion process, with no additional training. This paper proposes a simple invertible neural network architecture for this problem and presents our current findings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。