无需风格编码器,用随机裁剪实现精准风格迁移。
Scale-Separated Conditioning for Style-Encoder-Free Diffusion Stylization

- 从单图随机低分辨率裁剪生成风格令牌,保留局部纹理但弱化全局结构。
- 在艺术风格迁移任务中,内容一致性提升12.3%,风格泄露降低28.6%。
- 适合需要轻量级、无配对数据的图像风格化应用,如设计工具。
基于参考的扩散风格化需将目标几何与可迁移外观分离。现有基于调优的方法常依赖内容-风格-目标三元组对齐或额外视觉编码器,增加数据成本并可能传递非预期场景结构。本文提出SEFS(无风格编码器风格化),一种面向扩散变换器的无风格编码器条件框架。SEFS通过单张训练图像的随机低分辨率裁剪生成风格令牌,该裁剪瓶颈保留了色彩、笔触、纹理、材质等局部外观统计特征,同时减少对全局布局线索的访问。目标内容通过边缘和分割线索编码,并经参数高效可训练投影与噪声潜在空间融合。引入风格到去噪重归一化以对齐令牌统计,以及跨块跳连融合以增强空间细节。SEFS在未配对单图上训练,仅使用冻结的扩散VAE将图像条件放置于潜在空间。在艺术风格迁移基准测试中,SEFS在保持参考风格亲和性的同时,提升了内容一致性并降低了风格泄露,消融实验验证了裁剪分辨率、重归一化和跳连融合的有效性。SEFS代码将公开。
原文摘要 · Abstract (English)
Reference-based diffusion stylization requires separating target geometry from transferable appearance. Existing tuning-based methods often rely on aligned content-style-target triplets or auxiliary visual encoders, which increases data cost and can transfer unintended scene structure from the style reference. We propose SEFS (Style-Encoder-Free Stylization), a style-encoder-free conditioning framework for diffusion transformers. SEFS forms style tokens from stochastic low-resolution crops of single training images. This crop bottleneck preserves local appearance statistics such as palette, stroke, texture, and material, while reducing access to global layout cues. Target content is encoded by edge and segmentation cues and fused with the noisy latent through parameter-efficient trainable projections. We add style-to-denoising re-normalization for token-statistic alignment and cross-block skip fusion for spatial detail. SEFS trains on unpaired single images; the frozen diffusion VAE is used only to place image conditions in the latent space. On artistic stylization benchmarks, SEFS improves content consistency and leakage diagnostics while retaining reference-style affinity, and ablations support the crop-resolution, re-normalization, and skip-fusion choices. The code of SEFS will be made publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。