从一张图中分离出主体和风格，生成可独立修改的LoRA模型。
UnZipLoRA: Separating Content and Style from a Single Image
- 通过联合训练两个LoRA，从单张图像中解耦主体与风格。
- 解耦后的LoRA可直接相加，兼容性好且能还原原图。
- 适合需要灵活替换主体或风格的创意设计场景。
本文提出UnZipLoRA，一种从单张图像中分离主体与风格的方法，将二者表示为两个独立的低秩适配（LoRA）。与仅关注主体或风格、需分别训练的传统个性化方法不同，UnZipLoRA通过联合训练实现两者解耦，并确保生成的LoRA可无缝组合（直接相加）。该方法支持对主体和风格的独立操作，包括生成变体、将提取风格应用于新主体，以及重建原图或生成新组合。为解决主体与风格纠缠问题，引入新颖的提示分离技术，结合列与块级分离策略，精准保留特征并保证LoRA间的兼容性。人类评估与量化指标表明，其效果优于当前先进方法，如DreamBooth-LoRA、Inspiration Tree和B-LoRA。
原文摘要 · Abstract (English)
This paper introduces UnZipLoRA, a method for decomposing an image into its constituent subject and style, represented as two distinct LoRAs (Low-Rank Adaptations). Unlike existing personalization techniques that focus on either subject or style in isolation, or require separate training sets for each, UnZipLoRA disentangles these elements from a single image by training both the LoRAs simultaneously. UnZipLoRA ensures that the resulting LoRAs are compatible, i.e., they can be seamlessly combined using direct addition. UnZipLoRA enables independent manipulation and recontextualization of subject and style, including generating variations of each, applying the extracted style to new subjects, and recombining them to reconstruct the original image or create novel variations. To address the challenge of subject and style entanglement, UnZipLoRA employs a novel prompt separation technique, as well as column and block separation strategies to accurately preserve the characteristics of subject and style, and ensure compatibility between the learned LoRAs. Evaluation with human studies and quantitative metrics demonstrates UnZipLoRA's effectiveness compared to other state-of-the-art methods, including DreamBooth-LoRA, Inspiration Tree, and B-LoRA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。