让电商图像编辑像搭积木一样精准可控,支持多步局部修改。
GMO-E$^2$DIT: Grounded Multi-Operation Editing for E-Commerce Images

- 用视觉语言模型拆解指令,分步规划修改区域和操作
- 通过迭代反馈机制实现错误恢复与部分进度保留
- 专为电商设计,适合需要高精度、可审计的图像编辑场景
真实电商图像编辑常需多步、局部且可追溯的操作,而非全局重绘。现有单步编辑器将意图理解、空间定位与生成合成耦合,易因指令模糊导致部分失败,难以满足商业需求。为此,我们提出GMO-E$^2$DIT,一个由视觉语言模型驱动的智能编辑框架,将认知推理与生成渲染解耦。给定不明确指令时,VLM代理生成基于区域的编辑计划,并通过操作感知掩码与引用执行子任务,利用反射驱动循环检查中间结果并决定下一步。该迭代机制能可靠保留安全进度,重试未完成操作并恢复错误。我们还构建了统一数据流水线与EComEditBench基准,用于指导规划、执行与反思的联合监督评估。大量实验表明,GMO-E$^2$DIT在指令准确率与编辑保真度上均优于现有基线,表现媲美主流闭源模型。
原文摘要 · Abstract (English)
Real-world e-commerce image editing often requires multiple, localized, and auditable operations rather than global restyling. This compositional nature poses a dual challenge: models must precisely apply all requested edits to the correct regions while preserving unmodified content, even under ambiguous instructions. Existing one-shot editors conflate intent resolution, spatial grounding, and synthesis into a single step, frequently resulting in partial execution failures, which is unacceptable for commercial scenarios. To address this, we introduce GMO-E$^2$DIT, an agentic editing framework that couples a Vision-Language Model (VLM) with a mask-conditioned image editor to tackle structured multi-turn task completion. Given an underspecified instruction, the VLM agent constructs a region-grounded edit agenda, effectively decoupling cognitive reasoning from generative rendering. The framework then executes sub-programs via operation-aware masks and references, utilizing a reflection-driven loop to inspect intermediate results and determine the subsequent state. This iterative mechanism reliably preserves safe partial progress, retries unfinished operations, and recovers from errors. Furthermore, we develop a unified data pipeline providing aligned supervision for planning, execution, and reflection, alongside EComEditBench, a comprehensive benchmark for instruction-driven evaluation. Extensive experiments demonstrate that GMO-E$^2$DIT achieves competitive performance compared to strong closed-source models, yielding superior instruction accuracy and edit fidelity over existing baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。