arXiv:2605.04566cs.CVcs.CL2026-05被引 1

开源图像编辑模型无需训练就能完成视觉理解任务。

Open-Source Image Editing Models Are Zero-Shot Vision Learners

  • 用三个开源编辑模型直接测试零样本视觉能力,无任何微调。
  • 在NYUv2上表面法线误差仅17.69度,优于微调的Marigold。
  • 适合研究零样本视觉、模型泛化能力或可复现基准的开发者。

近期研究表明大型生成模型可在未显式训练的任务上完成视觉任务。然而,现有证据依赖闭源模型(如Veo 3、Nano Banana Pro)或需特定任务指令微调,尚不清楚公开可用的图像编辑模型是否具备开箱即用的零样本视觉能力。本文系统评估了三个开源图像编辑模型——Qwen-Image-Edit、FireRed-Image-Edit和LongCat-Image-Edit——在密集视觉预测任务上的表现,未进行任何微调。评测涵盖单目深度估计(在NYUv2和DIODE上)、表面法线估计(在NYUv2上)及语义分割(在Cityscapes上),覆盖几何与语义场景理解。结果表明,开源图像编辑模型展现出显著的零样本视觉理解能力:在NYUv2表面法线任务中,FireRed-Image-Edit达到17.69°平均角度误差,优于微调后的Marigold(20.86°),并接近指令微调的Vision Banana(17.78°);在NYUv2深度估计中,LongCat-Image-Edit取得δ₁=0.822(仿射对齐下),而Qwen-Image-Edit在DIODE Indoor上达δ₁=0.868;在Cityscapes语义分割中,Qwen-Image-Edit在19类上达25.7 mIoU,7类粗粒度下达49.5 mIoU。通过对比三款独立训练的编辑器,验证了零样本视觉能力是否为图像编辑预训练的涌现特性而非模型特异性现象。代码、评估脚本及全部结果已公开,为后续研究提供可复现基准。

原文摘要 · Abstract (English)

Recent studies have shown that large generative models can solve vision tasks they were not explicitly trained for. However, existing evidence relies on closed-source models~(Veo~3, Nano Banana Pro) or requires task-specific instruction tuning, leaving open whether publicly available image-editing models possess zero-shot vision abilities out of the box. We conduct a systematic evaluation of three open-source image-editing models -- Qwen-Image-Edit, FireRed-Image-Edit, and LongCat-Image-Edit -- on dense visual prediction tasks \emph{without any fine-tuning}. We benchmark monocular depth estimation on NYUv2 and DIODE, surface normal estimation on NYUv2, and semantic segmentation on Cityscapes, covering both geometric and semantic scene understanding. Results show that open-source image-editing models exhibit non-trivial zero-shot visual understanding. On NYUv2 surface normals, FireRed-Image-Edit achieves a mean angular error of $17.69^\circ$, surpassing the fine-tuned Marigold ($20.86^\circ$) and matching the instruction-tuned Vision Banana ($17.78^\circ$) without any task-specific training. On NYUv2 depth estimation, LongCat-Image-Edit obtains $δ_1{=}0.822$ with affine alignment, and Qwen-Image-Edit leads on DIODE Indoor ($δ_1{=}0.868$). On Cityscapes semantic segmentation, Qwen-Image-Edit reaches 25.7 mIoU at the 19-class level and 49.5 mIoU at a coarser 7-category level. By comparing three independently trained editors, we test whether zero-shot vision ability is an emergent property of image-editing pretraining rather than a model-specific artifact. Code, evaluation scripts, and all results are publicly released to serve as a reproducible baseline for future work.

零样本学习图像编辑视觉理解开源模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。