用大模型提升个性化文生图的风格一致性和内容准确性
LLM-Enabled Style and Content Regularization for Personalized Text-to-Image Generation
- 用视觉推理提示和参考图优化风格嵌入
- 保持模型泛化能力，解决内容偏差问题
- 适合需要高一致性个性图像生成的研究者
个性化文生图随着Stable Diffusion的出现迅速发展。现有方法通常通过嵌入标识符微调模型，但常因文本控制力下降导致风格不足和内容不准。本文提出风格优化与内容保留策略：前者利用视觉推理提示和参考图的语义信息优化风格嵌入，实现更精准一致的风格表达；后者通过保留模型泛化能力，增强文本控制力而不牺牲风格表现。实验验证，该方法在生成一致且个性化的图文输出上表现更优。
原文摘要 · Abstract (English)
The personalized text-to-image generation has rapidly advanced with the emergence of Stable Diffusion. Existing methods, which typically fine-tune models using embedded identifiers, often struggle with insufficient stylization and inaccurate image content due to reduced textual controllability. In this paper, we propose style refinement and content preservation strategies. The style refinement strategy leverages the semantic information of visual reasoning prompts and reference images to optimize style embeddings, allowing a more precise and consistent representation of style information. The content preservation strategy addresses the content bias problem by preserving the model's generalization capabilities, ensuring enhanced textual controllability without compromising stylization. Experimental results verify that our approach achieves superior performance in generating consistent and personalized text-to-image outputs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。