用一张图实现刺绣风格的精准定制,无需大量样本。
One-shot Embroidery Customization via Contrastive LoRA Modulation
- 基于对比学习分离图像风格与内容特征。
- 仅需单张参考图即可生成高质量刺绣效果。
- 适用于刺绣、上色、风格迁移等多种场景。
扩散模型显著提升了图像编辑能力,其生成逼真图像的能力正逐步改变零售预览流程。除了艺术风格迁移,细粒度视觉特征迁移也日益重要。刺绣作为复杂的纺织艺术,涉及多样针法与材质特性,现有风格迁移方法难以应对。为此,我们提出一种新颖的对比学习框架,仅需单张参考图即可解耦细粒度风格与内容特征,基于图像类比思想构建目标风格。首先构造图像对定义目标风格,再利用预训练扩散模型解耦表示设计相似性度量实现风格-内容分离。随后提出两阶段对比LoRA调制技术:第一阶段迭代更新完整LoRA及选定风格模块以初步分离;第二阶段通过自知识蒸馏策略进一步解耦。最终构建推理流程,仅使用风格模块即可处理图像或文本输入。为评估细粒度风格迁移能力,我们建立刺绣定制基准。所提方法在该任务上优于先前方法,并展现出对艺术风格迁移、草图着色和外观迁移三个额外领域的强泛化能力。
原文摘要 · Abstract (English)
Diffusion models have significantly advanced image manipulation techniques, and their ability to generate photorealistic images is beginning to transform retail workflows, particularly in presale visualization. Beyond artistic style transfer, the capability to perform fine-grained visual feature transfer is becoming increasingly important. Embroidery is a textile art form characterized by intricate interplay of diverse stitch patterns and material properties, which poses unique challenges for existing style transfer methods. To explore the customization for such fine-grained features, we propose a novel contrastive learning framework that disentangles fine-grained style and content features with a single reference image, building on the classic concept of image analogy. We first construct an image pair to define the target style, and then adopt a similarity metric based on the decoupled representations of pretrained diffusion models for style-content separation. Subsequently, we propose a two-stage contrastive LoRA modulation technique to capture fine-grained style features. In the first stage, we iteratively update the whole LoRA and the selected style blocks to initially separate style from content. In the second stage, we design a contrastive learning strategy to further decouple style and content through self-knowledge distillation. Finally, we build an inference pipeline to handle image or text inputs with only the style blocks. To evaluate our method on fine-grained style transfer, we build a benchmark for embroidery customization. Our approach surpasses prior methods on this task and further demonstrates strong generalization to three additional domains: artistic style transfer, sketch colorization, and appearance transfer.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。