arXiv:2601.02242cs.CVcs.AI2026-01被引 2

用小模型实现高质量图像编辑,4秒出图且不超显存。

VIBE: Visual Instruction Based Editor

  • 用20亿参数的视觉语言模型引导,16亿参数扩散模型生成。
  • 在多个评测中超越更大更贵模型,尤其擅长保留原图的修改。
  • 仅需24GB显存,4秒生成2K分辨率图像,适合轻量部署。

基于指令的图像编辑是生成式AI发展最快的领域之一。尽管过去一年涌现出大量开源与商用系统,但多数开源方案仍难以达到真实世界质量。主流扩散模型通常包含60亿至200亿参数,计算开销大。本文提出一种紧凑、高吞吐的指令式图像编辑流程:采用20亿参数的Qwen3-VL模型作为编辑指导,16亿参数的Sana1.5扩散模型生成图像。设计涵盖架构、数据处理、训练配置与评估,兼顾低成本推理与严格源一致性,同时保持高画质。在ImgEdit和GEdit基准上,该方法性能匹配或优于参数量多倍且推理成本更高的基线模型,尤其在属性调整、物体移除、背景修改和目标替换等需保留输入图像的任务中表现优异。模型仅占用24GB GPU内存,使用NVIDIA H100以BF16精度可在约4秒内生成2K分辨率图像,无需额外优化或蒸馏。

原文摘要 · Abstract (English)

Instruction-based image editing is among the fastest developing areas in generative AI. Over the past year, the field has reached a new level, with dozens of open-source models released alongside highly capable commercial systems. However, only a limited number of open-source approaches currently achieve real-world quality. In addition, diffusion backbones, the dominant choice for these pipelines, are often large and computationally expensive for many deployments and research settings, with widely used variants typically containing 6B to 20B parameters. This paper presents a compact, high-throughput instruction-based image editing pipeline that uses a modern 2B-parameter Qwen3-VL model to guide the editing process and the 1.6B-parameter diffusion model Sana1.5 for image generation. Our design decisions across architecture, data processing, training configuration, and evaluation target low-cost inference and strict source consistency while maintaining high quality across the major edit categories feasible at this scale. Evaluated on the ImgEdit and GEdit benchmarks, the proposed method matches or exceeds the performance of substantially heavier baselines, including models with several times as many parameters and higher inference cost, and is particularly strong on edits that require preserving the input image, such as an attribute adjustment, object removal, background edits, and targeted replacement. The model fits within 24 GB of GPU memory and generates edited images at up to 2K resolution in approximately 4 seconds on an NVIDIA H100 in BF16, without additional inference optimizations or distillation.

图像编辑轻量化模型扩散模型视觉语言

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。