让风格迁移更精准:通过语义解耦和先验蒸馏,保留细节与结构。
StyDeco: Unsupervised Style Transfer with Distilling Priors and Semantic Decoupling
- 用无监督方式蒸馏风格知识,生成伪配对数据训练模型。
- 通过对比聚类使文本编码器区分源与目标风格,提升语义一致性。
- 适合需要精细风格迁移与去风格化的研究者与开发者使用。
扩散模型已成为风格迁移的主流范式,但其依赖文本引导的机制存在根本局限:将文本描述视为统一整体,忽视了文本的非空间性与视觉风格的空间属性之间的语义鸿沟,常导致语义结构与细粒度细节在迁移中丢失。本文提出StyDeco,一种无监督框架,通过为风格迁移任务专门学习文本表示来解决此问题。首先,采用先验引导的数据蒸馏(PGD)策略,利用强大的冻结生成模型自动合成伪配对数据以蒸馏风格知识。随后,引入对比语义解耦(CSD),一种任务特定的目标函数,通过领域特异性权重调整文本编码器,在语义空间中执行双类聚类,促使源与目标表示形成独立聚类。在三个经典基准上的大量实验表明,该框架在风格保真度与结构保持方面均优于现有方法,验证了其在语义保持下的有效性。此外,该框架还支持独特的去风格化过程,进一步体现其可扩展性。代码已公开于https://github.com/QuanjianSong/StyDeco。
原文摘要 · Abstract (English)
Diffusion models have emerged as the dominant paradigm for style transfer, but their text-driven mechanism is hindered by a core limitation: it treats textual descriptions as uniform, monolithic guidance. This limitation overlooks the semantic gap between the non-spatial nature of textual descriptions and the spatially-aware attributes of visual style, often leading to the loss of semantic structure and fine-grained details during stylization. In this paper, we propose StyDeco, an unsupervised framework that resolves this limitation by learning text representations specifically tailored for the style transfer task. Our framework first employs Prior-Guided Data Distillation (PGD), a strategy designed to distill stylistic knowledge without human supervision. It leverages a powerful frozen generative model to automatically synthesize pseudo-paired data. Subsequently, we introduce Contrastive Semantic Decoupling (CSD), a task-specific objective that adapts a text encoder using domain-specific weights. CSD performs a two-class clustering in the semantic space, encouraging source and target representations to form distinct clusters. Extensive experiments on three classic benchmarks demonstrate that our framework outperforms several existing approaches in both stylistic fidelity and structural preservation, highlighting its effectiveness in style transfer with semantic preservation. In addition, our framework supports a unique de-stylization process, further demonstrating its extensibility. Our code is vailable at https://github.com/QuanjianSong/StyDeco.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。