无需艺术数据预训练，仅用少量样本即可让模型学会艺术风格。
Opt-In Art: Learning Art Styles Only from Few Examples
- 纯照片训练模型，通过少量艺术样本后适应生成艺术风格。
- 生成效果与大规模艺术数据训练的模型相当，用户与自动评估均验证。
- 发现艺术风格可“按需引入”，无需提前接触艺术数据。
我们探讨了模型在未接触绘画数据的情况下，能否仅凭少量示例学习艺术风格。为此，我们仅用照片训练文本到图像模型，不包含任何与绘画相关的内容。结果表明，即使未经艺术数据预训练，模型仍可通过少量艺术样本实现风格迁移。用户研究和自动评估均显示，微调后的模型性能达到与大量含绘画、素描或插图数据集训练的先进模型相当水平。最后，借助数据归因技术分析发现，高质量艺术图像可不依赖前期艺术数据生成，说明艺术风格可在可控的“按需”模式下，仅通过有限且精心挑选的样本实现。
原文摘要 · Abstract (English)
We explore whether pre-training on datasets with paintings is necessary for a model to learn an artistic style with only a few examples. To investigate this, we train a text-to-image model exclusively on photographs, without access to any painting-related content. We show that it is possible to adapt a model that is trained without paintings to an artistic style, given only few examples. User studies and automatic evaluations confirm that our model (post-adaptation) performs on par with state-of-the-art models trained on massive datasets that contain artistic content like paintings, drawings or illustrations. Finally, using data attribution techniques, we analyze how both artistic and non-artistic datasets contribute to generating artistic-style images. Surprisingly, our findings suggest that high-quality artistic outputs can be achieved without prior exposure to artistic data, indicating that artistic style generation can occur in a controlled, opt-in manner using only a limited, carefully selected set of training examples.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。