让图像编辑辅助大模型推理,提升细粒度视觉理解能力。
ETCHR: Editing To Clarify and Harness Reasoning

- 用可条件编辑的图像模型分离推理与理解,针对性解决语言与生成双短板。
- 在五类任务中平均准确率提升4.6%~5.5%,最高达81.16%。
- 无需训练即可接入多种主流多模态大模型,适合需要精细视觉推理的场景。
多模态大语言模型虽已提升视觉推理能力,但纯文本思维链仍是细粒度聚焦或视角变换任务的瓶颈。'以图思考'范式虽缩小了差距,但现有方法受限于固定工具包或统一模型生成噪声中间图像。本文提出第三种方案:使用专用图像编辑模型,并与理解模型解耦。然而现成编辑器存在双重缺陷:语言侧无法将抽象问题映射为恰当视觉变换,生成侧随推理深度增加而失真。基于此分析,我们提出ETCHR(Editing To Clarify and Harness Reasoning),一种基于问题条件、具备推理感知的图像编辑器,采用两阶段训练策略:先通过监督微调学习编辑轨迹,再利用视觉语言模型奖励增强编辑正确性与下游推理准确率。由于解耦设计,ETCHR可无训练地适配各类开源与闭源多模态大模型。在五个任务类别(细粒度感知、图表理解、逻辑推理、拼图复原、3D理解)中,其平均Pass@1分别从55.95提升至60.77(+4.82,Qwen3-VL-8B)、65.08至70.55(+5.47,Gemini-3.1-Flash-Lite)、76.55至81.16(+4.61,Kimi K2.5)。
原文摘要 · Abstract (English)
Multimodal Large Language Models have advanced visual reasoning, yet a purely textual chain of thought remains a bottleneck for questions that require fine-grained focus or view transformations. The ''think with images'' paradigm narrows this gap, but existing approaches are either constrained by fixed predefined toolkits or produce noisy intermediate images from unified multimodal methods. We pursue a third option: using a dedicated image editing model and decouple it with an understanding model. However, off-the-shelf image editors fail as reasoning assistants with two complementary gaps: a language-side gap, where editors trained as passive instruction-followers cannot map an abstract question to an appropriate visual transformation, and a generation-side gap, where edit correctness degrades as reasoning depth grows. Guided by this analysis, we introduce ETCHR (Editing To Clarify and Harness Reasoning), a question-conditioned, reasoning-aware image editor decoupled from the downstream understanding model and trained with a two-stage recipe targeted at the two gaps: Reasoning Imitation via supervised fine-tuning on edit trajectories, followed by Reasoning Enhancement with VLM-derived rewards for edit correctness and downstream reasoning accuracy. Since the editor is decoupled, ETCHR plugs into different open- and closed-source MLLMs in a training-free manner. Across five task families (fine-grained perception, chart understanding, logic reasoning, jigsaw restoration, and 3D understanding), ETCHR raises average Pass@1 from 55.95 to 60.77 (+4.82) with Qwen3-VL-8B, from 65.08 to 70.55 (+5.47) with Gemini-3.1-Flash-Lite, and from 76.55 to 81.16 (+4.61) with the 1T-parameter MoE model Kimi K2.5.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。