用一个数字代码就能生成独特且一致的图像风格。
A Style is Worth One Code: Unlocking Code-to-Style Image Generation with Discrete Style Space
- 用离散风格码本提取风格嵌入,通过数值代码控制生成
- 仅凭一个数字代码即可生成多样且可复现的新风格图像
- 适合想快速试验风格、做艺术创作的开发者和设计师
创新性视觉风格化是艺术创作的核心,但生成新颖且一致的视觉风格仍是重大挑战。现有生成方法通常依赖长文本提示、参考图像或参数高效微调来引导风格感知图像生成,但常面临风格不一致、创造力有限和风格表示复杂等问题。本文提出‘代码到风格’图像生成新任务,证明一个风格值得一个数值代码。为此,我们提出首个开源方法CoTyle:首先从图像集合中训练离散风格码本提取风格嵌入;再训练自回归风格生成器建模嵌入分布,以合成新风格嵌入;推理时,将数值代码映射为唯一风格嵌入,驱动文生图扩散模型(T2I-DM)生成对应风格图像。相比现有方法,该方法实现前所未有的简洁与多样性,从极简输入解锁可复现的广阔风格空间。大量实验验证了CoTyle能有效将数字代码转化为风格控制器,证实‘一个风格值一个代码’。
原文摘要 · Abstract (English)
Innovative visual stylization is a cornerstone of artistic creation, yet generating novel and consistent visual styles remains a significant challenge. Existing generative approaches typically rely on lengthy textual prompts, reference images, or parameter-efficient fine-tuning to guide style-aware image generation, but often struggle with style consistency, limited creativity, and complex style representations. In this paper, we affirm that a style is worth one numerical code by introducing the novel task, code-to-style image generation, which produces images with novel, consistent visual styles conditioned solely on a numerical style code. To date, this field has only been primarily explored by the industry (e.g., Midjourney), with no open-source research from the academic community. To fill this gap, we propose CoTyle, the first open-source method for this task. Specifically, we first train a discrete style codebook from a collection of images to extract style embeddings. These embeddings serve as conditions for a text-to-image diffusion model (T2I-DM) to generate stylistic images. Subsequently, we train an autoregressive style generator on the discrete style embeddings to model their distribution, allowing the synthesis of novel style embeddings. During inference, a numerical style code is mapped to a unique style embedding by the style generator, and this embedding guides the T2I-DM to generate images in the corresponding style. Unlike existing methods, our method offers unparalleled simplicity and diversity, unlocking a vast space of reproducible styles from minimal input. Extensive experiments validate that CoTyle effectively turns a numerical code into a style controller, demonstrating a style is worth one code.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。