用大模型直接生成人体姿态关键点,实现精准对齐文本描述的图像生成。
PointT2I: LLM-based text-to-image generation via keypoints
- 通过大模型解析文本直接生成人体姿态关键点
- 结合关键点与文本生成符合姿态的高质量图像
- 无需微调,适合需要精确人体姿态控制的场景
文本到图像(T2I)生成模型已取得显著进展,能生成与输入提示高度一致的高质量图像。然而,当提示包含复杂概念(尤其是人体姿态)时,现有方法仍难以准确生成对应图像。本文提出PointT2I框架,利用大语言模型(LLM)实现无需微调的、基于关键点引导的图像生成。该框架包含三个组件:关键点生成、图像生成和反馈系统。关键点生成模块仅依赖输入提示,由LLM直接生成对应人体姿态的关键点;图像生成模块结合文本提示与生成的关键点,生成准确反映目标姿态的图像;反馈系统则利用LLM评估生成内容与提示之间的语义一致性,进一步优化输出。PointT2I是首个不依赖微调、仅凭文本提示实现关键点引导图像生成的方案,可精准生成与姿态描述一致的图像。
原文摘要 · Abstract (English)
Text-to-image (T2I) generation model has made significant advancements, resulting in high-quality images aligned with an input prompt. However, despite T2I generation's ability to generate fine-grained images, it still faces challenges in accurately generating images when the input prompt contains complex concepts, especially human pose. In this paper, we propose PointT2I, a framework that effectively generates images that accurately correspond to the human pose described in the prompt by using a large language model (LLM). PointT2I consists of three components: Keypoint generation, Image generation, and Feedback system. The keypoint generation uses an LLM to directly generate keypoints corresponding to a human pose, solely based on the input prompt, without external references. Subsequently, the image generation produces images based on both the text prompt and the generated keypoints to accurately reflect the target pose. To refine the outputs of the preceding stages, we incorporate an LLM-based feedback system that assesses the semantic consistency between the generated contents and the given prompts. Our framework is the first approach to leveraging LLM for keypoints-guided image generation without any fine-tuning, producing accurate pose-aligned images based solely on textual prompts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。