arXiv:2603.29602cs.GRcs.AI2026-03被引 2

通过规划-执行-反思闭环,实现多轮图像编辑的精准控制与自修正。

IMAGAgent: Orchestrating Multi-Turn Image Editing via Constraint-Aware Planning and Reflection

  • 引入约束感知规划,将复杂指令拆解为可执行的原子任务。
  • 动态调度异构工具链,结合上下文实现自适应编辑路径。
  • 多专家协同反思机制,自动纠错并优化后续决策过程。

现有多轮图像编辑方法通常局限于孤立的单步执行,因缺乏上下文感知和闭环反馈机制,易导致错误累积与语义漂移,最终引发生成图像严重结构失真。为此,我们提出IMAGAgent,一种基于“规划-执行-反思”闭环机制的多轮图像编辑代理框架,在统一流程中实现指令解析、工具调度与自适应修正的深度协同。首先,设计约束感知规划模块,利用视觉语言模型(VLM)将复杂自然语言指令精确分解为一系列受目标唯一性、语义原子性和视觉可感知性约束的可执行子任务。其次,工具链编排模块根据当前图像、子任务及历史上下文动态构建执行路径,实现跨图像检索、分割、检测与编辑等异构模型的自适应调度与协同操作。最后,提出多专家协同反思机制,由中心大语言模型(LLM)接收待编辑图像,并整合VLM的批判意见生成整体反馈,同时触发细粒度自我修正,并记录反馈结果以优化未来决策。在自建的MTEditBench与MagicBrush数据集上的大量实验表明,IMAGAgent在指令一致性、编辑精度与整体质量上均显著优于现有方法。代码已开源:https://github.com/hackermmzz/IMAGAgent.git。

原文摘要 · Abstract (English)

Existing multi-turn image editing paradigms are often confined to isolated single-step execution. Due to a lack of context-awareness and closed-loop feedback mechanisms, they are prone to error accumulation and semantic drift during multi-turn interactions, ultimately resulting in severe structural distortion of the generated images. For that, we propose \textbf{IMAGAgent}, a multi-turn image editing agent framework based on a "plan-execute-reflect" closed-loop mechanism that achieves deep synergy among instruction parsing, tool scheduling, and adaptive correction within a unified pipeline. Specifically, we first present a constraint-aware planning module that leverages a vision-language model (VLM) to precisely decompose complex natural language instructions into a series of executable sub-tasks, governed by target singularity, semantic atomicity, and visual perceptibility. Then, the tool-chain orchestration module dynamically constructs execution paths based on the current image, the current sub-task, and the historical context, enabling adaptive scheduling and collaborative operation among heterogeneous operation models covering image retrieval, segmentation, detection, and editing. Finally, we devise a multi-expert collaborative reflection mechanism where a central large language model (LLM) receives the image to be edited and synthesizes VLM critiques into holistic feedback, simultaneously triggering fine-grained self-correction and recording feedback outcomes to optimize future decisions. Extensive experiments on our constructed \textbf{MTEditBench} and the MagicBrush dataset demonstrate that IMAGAgent achieves performance significantly superior to existing methods in terms of instruction consistency, editing precision, and overall quality. The code is available at https://github.com/hackermmzz/IMAGAgent.git.

多轮编辑图像生成智能代理闭环控制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。