通过合成数据训练,让图像编辑模型更稳定地分解复杂指令。
Towards Robust Sequential Decomposition for Complex Image Editing

- 用合成数据构建多层级编辑任务,学习如何拆解复杂指令
- 在复杂任务下,分解策略比单步编辑更准确且减少错误累积
- 模型可从模拟数据迁移到真实图像,适合复杂图像编辑场景
视觉生成模型虽能实现高保真图像编辑,但在处理包含组合操作或步骤依赖的复杂指令时仍表现不佳。这源于两种主流范式的问题:单步编辑难以准确解析复杂指令,导致误编辑;而顺序编辑虽可分解任务,但执行过程中的误差会逐层累积,影响结果质量。本文在统一的上下文编辑框架下分析不同范式的编辑行为,提出一种合成数据生成管道,构建了大规模、高质量的复杂编辑数据集。通过在合成数据上微调,发现合理设计的顺序分解策略可在任务复杂度提升时依然保持鲁棒性。进一步地,结合真实世界数据联合训练,模型学习到的分解能力可有效迁移到真实图像,验证了模拟到真实泛化的潜力。
原文摘要 · Abstract (English)
Recent advances in visual generative models have enabled high-fidelity image editing guided by human instructions. However, these models often struggle with complex instructions involving combinatorial editing operations or inter-step dependencies. This difficulty stems from the limitations of two canonical paradigms: (1) single-turn editing, which attempts to apply all instructed edits in one pass, often fails to parse the complex instruction accurately and causes undesired edits; and (2) sequential editing can decompose the task into simpler steps but suffers from compounding errors introduced by the sequential execution, leading to low-fidelity results. To derive a robust solution for complex image editing, we examine editing behaviors of different paradigms under a unified in-context editing framework, and study how the benefits of sequential decomposition can be balanced against its error-accumulation drawbacks. We further develop a synthetic data pipeline that constructs editing tasks of varying instruction complexity, allowing us to curate a large-scale editing dataset with high-quality decomposed sequences. By finetuning on synthetic data, we discovered that with properly designed editing paradigms, sequential decomposition yields robust improvements even as task complexity increases. Furthermore, the decomposition skills learned from synthetic tasks can transfer to real images by co-training with real-world editing data, demonstrating the promise of sim-to-real generalization for tackling complex image editing across broader domains.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。