arXiv:2505.06176cs.GRcs.CV2025-05中稿 · SIGGRAPH被引 12

让大模型通过解谜学会图像修图,既懂原理又可解释。

MonetGPT: Solving Puzzles Enhances MLLMs' Image Retouching Skills

  • 用视觉谜题训练大模型理解图像操作原理
  • 能规划并生成可解释的修图步骤序列
  • 适合需要可控、保真修图的设计师和用户

照片修图是后期处理的重要任务。生成式编辑虽易用,却常不可控地改变物体原貌;而传统程序化编辑(如 Gimp、Lightroom 支持)虽安全,对新手却难以规划。本文提出 MonetGPT,让多模态大模型通过解特制视觉谜题,先学习底层图像操作逻辑,再基于专家修图数据,合成推理数据集,微调模型以规划可解释的编辑序列。所生成操作保留对象细节与分辨率,且可被用户覆盖。在多个测试案例中,该方法在可解释性与身份保持方面优于现有生成式及程序化方案。

原文摘要 · Abstract (English)

Retouching is an essential task in post-manipulation of raw photographs. Generative editing, guided by text or strokes, provides a new tool accessible to users but can easily change the identity of the original objects in unacceptable and unpredictable ways. In contrast, although traditional procedural edits, as commonly supported by photoediting tools (e.g., Gimp, Lightroom), are conservative, they are still preferred by professionals. Unfortunately, professional quality retouching involves many individual procedural editing operations that is challenging to plan for most novices. In this paper, we ask if a multimodal large language model (MLLM) can be taught to critique raw photographs, suggest suitable remedies, and finally realize them with a given set of pre-authored procedural image operations. We demonstrate that MLLMs can be first made aware of the underlying image processing operations, by training them to solve specially designed visual puzzles. Subsequently, such an operation-aware MLLM can both plan and propose edit sequences. To facilitate training, given a set of expert-edited photos, we synthesize a reasoning dataset by procedurally manipulating the expert edits and then grounding a pretrained LLM on the visual adjustments, to synthesize reasoning for finetuning. The proposed retouching operations are, by construction, understandable by the users, preserve object details and resolution, and can be optionally overridden. We evaluate our setup on a variety of test examples and show advantages, in terms of explainability and identity preservation, over existing generative and other procedural alternatives. Code, data, models, and supplementary results can be found via our project website at https://monetgpt.github.io.

图像修图多模态模型可控生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。