统一风格与主体生成,实现风格相似与主体一致的兼顾
USO: Unified Style and Subject-Driven Generation via Disentangled and Reward Learning
- 通过解耦学习分离内容与风格特征,双目标协同优化
- 在多个指标上超越开源模型,在风格相似与主体一致上均达顶尖水平
- 首次提供联合评估基准,适合风格迁移与个性化生成研究者
现有研究通常将风格驱动与主体驱动生成视为两个独立任务:前者强调风格相似性,后者要求主体一致性,导致明显矛盾。我们认为二者可在统一框架下实现统一,因其本质均涉及内容与风格的解耦与重组,这是风格生成研究的核心主题。为此,我们提出USO——一种统一风格-主体优化定制模型。首先构建大规模三元组数据集,包含内容图、风格图及其对应风格化图像。其次引入解耦学习机制,通过风格对齐训练和内容-风格解耦训练双重目标,同步实现风格特征对齐与内容-风格分离。第三,采用风格奖励学习(SRL)进一步提升性能。最后,发布USO-Bench,首个联合评估风格相似性与主体保真度的基准。大量实验表明,USO在开源模型中于主体一致性和风格相似性两方面均达到当前最优表现。代码与模型:https://github.com/bytedance/USO
原文摘要 · Abstract (English)
Existing literature typically treats style-driven and subject-driven generation as two disjoint tasks: the former prioritizes stylistic similarity, whereas the latter insists on subject consistency, resulting in an apparent antagonism. We argue that both objectives can be unified under a single framework because they ultimately concern the disentanglement and re-composition of content and style, a long-standing theme in style-driven research. To this end, we present USO, a Unified Style-Subject Optimized customization model. First, we construct a large-scale triplet dataset consisting of content images, style images, and their corresponding stylized content images. Second, we introduce a disentangled learning scheme that simultaneously aligns style features and disentangles content from style through two complementary objectives, style-alignment training and content-style disentanglement training. Third, we incorporate a style reward-learning paradigm denoted as SRL to further enhance the model's performance. Finally, we release USO-Bench, the first benchmark that jointly evaluates style similarity and subject fidelity across multiple metrics. Extensive experiments demonstrate that USO achieves state-of-the-art performance among open-source models along both dimensions of subject consistency and style similarity. Code and model: https://github.com/bytedance/USO
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。