arXiv:2604.00849cs.CV2026-04中稿 · CVPR

分离文本与视觉信息,再智能重组,解决图像生成中保真与可控的矛盾。

Disentangling to Re-couple: Resolving the Similarity-Controllability Paradox in Subject-Driven Text-to-Image Generation

  • 先分离主体身份与修改指令,用参考图提取主体,文本仅保留操作命令
  • 通过强化学习引入奖励机制,实现主体与上下文自然融合,保持高保真度
  • 适合需要精准控制场景但又不丢失主体特征的图像生成任务

主体驱动的文本到图像生成旨在根据文本提示改变场景的同时保持主体身份一致。核心挑战是‘保真-可控性悖论’:提升文本控制力常导致主体失真,反之亦然。我们指出该问题源于文本提示角色模糊——同时描述主体和修改内容,造成模型信号冲突。为此提出DisCo框架,先解耦后重耦:通过文本-视觉解耦模块,仅从参考图和主体词中提取主体身份,文本提示简化为仅含操作指令(使用泛指代词),消除描述歧义;但严格分离易导致组合不自然,因此设计专属奖励信号,结合强化学习实现视觉定义的主体与文本生成的上下文无缝重耦。实验表明,该方法有效化解悖论,在保留高保真主体的同时实现精确文本控制,达到当前最优性能,生成图像高度真实且连贯。

原文摘要 · Abstract (English)

Subject-Driven Text-to-Image (T2I) Generation aims to preserve a subject's identity while editing its context based on a text prompt. A core challenge in this task is the "similarity-controllability paradox", where enhancing textual control often degrades the subject's fidelity, and vice-versa. We argue this paradox stems from the ambiguous role of text prompts, which are often tasked with describing both the subject and the desired modifications, leading to conflicting signals for the model. To resolve this, we propose DisCo, a novel framework that first Disntangles and then re-Couples visual and textual information. First, our textual-visual decoupling module isolates the sources of information: subject identity is extracted exclusively from the reference image with the entity word of the subject, while the text prompt is simplified to contain only the modification command, where the subject refers to general pronouns, eliminating descriptive ambiguity. However, this strict separation can lead to unnatural compositions between the subject and its contexts. We address this by designing a dedicated reward signal and using reinforcement learning to seamlessly recouple the visually-defined subject and the textually-generated context. Our approach effectively resolves the paradox, enabling simultaneous high-fidelity subject preservation and precise textual control. Extensive experiments demonstrate that our method achieves state-of-the-art performance, producing highly realistic and coherent images.

图像生成文本控制解耦强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。