仅用一个提示词实现连贯图像生成,无需额外训练。
One-Prompt-One-Story: Free-Lunch Consistent Text-to-Image Generation Using a Single Prompt
- 将所有提示拼接成单个输入,利用语言模型上下文一致性保持角色一致。
- 提出奇异值重加权与身份保持交叉注意力,提升每帧生成准确度。
- 无需训练或修改模型,适配多种扩散模型和应用场景。
文本到图像生成模型能从输入提示生成高质量图像,但难以满足叙事中身份一致性的要求。现有方法通常需要大规模数据训练或对模型结构进行额外修改,限制了其在不同领域和多样扩散模型配置中的适用性。本文首次观察到语言模型具备通过单个提示理解身份的内在能力——上下文一致性。受此启发,我们提出一种无需训练的新型方法「One-Prompt-One-Story」(1Prompt1Story),将所有提示合并为单一输入,初始保留角色身份。随后通过两项新技术:奇异值重加权与身份保持交叉注意力,优化生成过程,确保每帧输出与描述高度一致。实验对比多种现有方法,在定量指标和定性评估中均证明其有效性。代码已开源:https://github.com/byliutao/1Prompt1Story。
原文摘要 · Abstract (English)
Text-to-image generation models can create high-quality images from input prompts. However, they struggle to support the consistent generation of identity-preserving requirements for storytelling. Existing approaches to this problem typically require extensive training in large datasets or additional modifications to the original model architectures. This limits their applicability across different domains and diverse diffusion model configurations. In this paper, we first observe the inherent capability of language models, coined context consistency, to comprehend identity through context with a single prompt. Drawing inspiration from the inherent context consistency, we propose a novel training-free method for consistent text-to-image (T2I) generation, termed "One-Prompt-One-Story" (1Prompt1Story). Our approach 1Prompt1Story concatenates all prompts into a single input for T2I diffusion models, initially preserving character identities. We then refine the generation process using two novel techniques: Singular-Value Reweighting and Identity-Preserving Cross-Attention, ensuring better alignment with the input description for each frame. In our experiments, we compare our method against various existing consistent T2I generation approaches to demonstrate its effectiveness through quantitative metrics and qualitative assessments. Code is available at https://github.com/byliutao/1Prompt1Story.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。