让图像中主体与背景分别精准换风格，避免风格错乱。
ObjMST: An Object-Focused Multimodal Style Transfer Framework
- 分主体和背景分别控制风格，用新损失函数保证风格对齐。
- 主体风格独立提取并融合，输出画面自然无拼接痕迹。
- 适合需要精细风格控制的图像编辑、创意设计场景。
我们提出 ObjMST，一种面向对象的多模态风格迁移框架，为显著物体与周围元素提供独立的风格监督，解决多模态表示学习中的对齐问题。现有图像-文本多模态风格迁移方法存在两大挑战：(1) 生成非对齐且不一致的多模态风格表征；(2) 内容错配，即相同风格模式同时应用于显著物体及其周围环境。我们的方法通过：(1) 引入风格特异性掩码方向性 CLIP 损失，确保显著物体及其周围区域风格表征的一致性和对齐性；(2) 采用显著物到关键元素映射机制对显著物体进行风格化，再通过图像调和实现风格化物体与环境的无缝融合。我们在定量指标和定性视觉评估上验证了 ObjMST 的有效性。代码已公开于 https://github.com/chandagrover/ObjMST。
原文摘要 · Abstract (English)
We propose ObjMST, an object-focused multimodal style transfer framework that provides separate style supervision for salient objects and surrounding elements while addressing alignment issues in multimodal representation learning. Existing image-text multimodal style transfer methods face the following challenges: (1) generating non-aligned and inconsistent multimodal style representations; and (2) content mismatch, where identical style patterns are applied to both salient objects and their surrounding elements. Our approach mitigates these issues by: (1) introducing a Style-Specific Masked Directional CLIP Loss, which ensures consistent and aligned style representations for both salient objects and their surroundings; and (2) incorporating a salient-to-key mapping mechanism for stylizing salient objects, followed by image harmonization to seamlessly blend the stylized objects with their environment. We validate the effectiveness of ObjMST through experiments, using both quantitative metrics and qualitative visual evaluations of the stylized outputs. Our code is available at: https://github.com/chandagrover/ObjMST.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。