解决情感图像生成中的情绪误用问题,让图像更真实表达情感。
Emotion-Director: Bridging Affective Shortcut in Emotion-Oriented Image Generation
- 用视觉与文本协同引导生成情感图像,突破语义局限。
- 引入负向视觉提示提升模型对相同语义下不同情绪的敏感度。
- 多智能体重写提示词,模拟人类情感表达,增强画面感染力。
基于扩散模型的图像生成展现出强大能力,推动了多样化应用的发展。由于情感在广告中的重要性,面向情感的图像生成受到越来越多关注。然而,现有方法存在情感捷径问题,即情感被简化为语义。根据二十年研究,情感不等同于语义。为此,我们提出Emotion-Director,一个跨模态协作框架,包含两个模块:首先,提出跨模态协同扩散模型(MC-Diffusion),融合视觉与文本提示进行引导,实现超越语义的情感图像生成;进一步,通过负向视觉提示改进DPO优化,增强模型在相同语义下对不同情感的敏感性。其次,提出MC-Agent,一个跨模态协作智能体系统,重写文本提示以传达目标情感。为避免模板化重写,MC-Agent采用多智能体模拟人类情感主观性,并通过概念链流程提升重写提示的视觉表现力。大量定性和定量实验验证了Emotion-Director在情感导向图像生成上的优越性。
原文摘要 · Abstract (English)
Image generation based on diffusion models has demonstrated impressive capability, motivating exploration into diverse and specialized applications. Owing to the importance of emotion in advertising, emotion-oriented image generation has attracted increasing attention. However, current emotion-oriented methods suffer from an affective shortcut, where emotions are approximated to semantics. As evidenced by two decades of research, emotion is not equivalent to semantics. To this end, we propose Emotion-Director, a cross-modal collaboration framework consisting of two modules. First, we propose a cross-Modal Collaborative diffusion model, abbreviated as MC-Diffusion. MC-Diffusion integrates visual prompts with textual prompts for guidance, enabling the generation of emotion-oriented images beyond semantics. Further, we improve the DPO optimization by a negative visual prompt, enhancing the model's sensitivity to different emotions under the same semantics. Second, we propose MC-Agent, a cross-Modal Collaborative Agent system that rewrites textual prompts to express the intended emotions. To avoid template-like rewrites, MC-Agent employs multi-agents to simulate human subjectivity toward emotions, and adopts a chain-of-concept workflow that improves the visual expressiveness of the rewritten prompts. Extensive qualitative and quantitative experiments demonstrate the superiority of Emotion-Director in emotion-oriented image generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。