通过分层思维链提升图像编辑的精细度与泛化能力
Meta-CoT: Enhancing Granularity and Generalization in Image Editing
- 将编辑任务拆解为任务-目标-理解能力三元组，提升模型理解精度
- 仅用五个元任务训练，实现对未见编辑任务15.8%的性能提升
- 引入推理-编辑一致性奖励，让思维链更有效指导生成
统一的多模态理解/生成模型通过将细粒度理解融入思维链（CoT）过程，提升了图像编辑性能。然而，如何设计有效的CoT形式与训练策略以同时增强理解精细度与泛化能力仍待探索。为此，我们提出Meta-CoT，一种双层分解范式：第一层将任意单图编辑操作分解为三元组（任务，目标，所需理解能力），通过生成任务特定的CoT并遍历所有目标，显著提升模型对编辑操作的理解粒度；第二层将编辑任务进一步拆分为五个基础元任务，实验表明在这些元任务上训练，结合三元组其他两要素，即可实现跨多样未见任务的强泛化。为对齐推理与编辑行为，我们引入CoT-Editing一致性奖励，促进模型更准确地利用思维链信息。实验显示，该方法在21个编辑任务上整体提升15.8%，且仅需少量元任务训练即可泛化至未见任务。代码、基准与模型已公开。
原文摘要 · Abstract (English)
Unified multi-modal understanding/generative models have shown improved image editing performance by incorporating fine-grained understanding into their Chain-of-Thought (CoT) process. However, a critical question remains underexplored: what forms of CoT and training strategy can jointly enhance both the understanding granularity and generalization? To address this, we propose Meta-CoT, a paradigm that performs a two-level decomposition of any single-image editing operation with two key properties: (1) Decomposability. We observe that any editing intention can be represented as a triplet - (task, target, required understanding ability). Inspired by this, Meta-CoT decomposes both the editing task and the target, generating task-specific CoT and traversing editing operations on all targets. This decomposition enhances the model's understanding granularity of editing operations and guides it to learn each element of the triplet during training, substantially improving the editing capability. (2) Generalizability. In the second decomposition level, we further break down editing tasks into five fundamental meta-tasks. We find that training on these five meta-tasks, together with the other two elements of the triplet, is sufficient to achieve strong generalization across diverse, unseen editing tasks. To further align the model's editing behavior with its CoT reasoning, we introduce the CoT-Editing Consistency Reward, which encourages more accurate and effective utilization of CoT information during editing. Experiments demonstrate that our method achieves an overall 15.8% improvement across 21 editing tasks, and generalizes effectively to unseen editing tasks when trained on only a small set of meta-tasks. Our code, benchmark, and model are released at https://shiyi-zh0408.github.io/projectpages/Meta-CoT/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。