用强化学习提升文本引导图像编辑的准确率，仅用少量数据就超越主流方法。
The Promise of RL for Autoregressive Image Editing
- 基于自回归多模态模型，结合强化学习与大模型验证器优化编辑过程。
- 在多种编辑任务中表现优于基线，训练数据量仅为基准的1/10左右。
- 适合需要高精度图文编辑、关注少样本训练的开发者和研究者。
尽管图像生成技术已能根据多句提示生成高质量图像，但文本引导的图像编辑仍面临挑战。即使仅需修改几个词的请求也常失败。本文探索三种策略：监督微调（SFT）、强化学习（RL）和思维链（CoT）推理。为统一评估，采用处理文本与视觉标记的自回归多模态模型。实验发现，结合大型多模态语言模型验证器的强化学习最为有效。因此，我们发布EARL：基于自回归与强化学习的图像编辑模型，在多样化的编辑任务上表现优异，且训练数据远少于现有方法。该模型推动了自回归多模态模型在图像编辑领域的边界。代码、训练数据及模型已开源至https://github.com/mair-lab/EARL。
原文摘要 · Abstract (English)
While image generation techniques are now capable of producing high-quality images that respect prompts which span multiple sentences, the task of text-guided image editing remains a challenge. Even edit requests that consist of only a few words often fail to be executed correctly. We explore three strategies to enhance performance on a wide range of image editing tasks: supervised fine-tuning (SFT), reinforcement learning (RL), and Chain-of-Thought (CoT) reasoning. In order to study all these components in one consistent framework, we adopt an autoregressive multimodal model that processes textual and visual tokens in a unified manner. We find RL combined with a large multi-modal LLM verifier to be the most effective of these strategies. As a result, we release EARL: Editing with Autoregression and RL, a strong RL-based image editing model that performs competitively on a diverse range of edits compared to strong baselines, despite using much less training data. Thus, EARL pushes the frontier of autoregressive multimodal models on image editing. We release our code, training data, and trained models at https://github.com/mair-lab/EARL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。