用视觉类比让模型像人一样看图推理，支持多种图像任务
VIRAL: Visual In-Context Reasoning via Analogy in Diffusion Transformers
- 把视觉上下文学习建模为图像类比生成，通过多图条件控制推理
- 在多个视觉任务上超越现有方法，开放域编辑也表现良好
- 新数据集覆盖感知、修复、编辑，适合研究视觉推理的学者
在计算机视觉中复现上下文学习（ICL）仍面临任务多样性挑战。我们提出VIRAL框架，通过将ICL建模为基于视觉类比的条件生成（x_s : x_t :: x_q : y_q），从预训练图像编辑模型中激发视觉推理能力。采用角色感知的多图像条件化方式适配冻结的扩散变压器（DiT），并引入专家混合型LoRA以缓解跨任务梯度干扰。此外，为弥补现有视觉上下文数据集的不足，我们构建了一个大规模数据集，涵盖感知、修复和编辑任务。实验表明，VIRAL在多数视觉任务上优于现有方法，验证了统一的视觉-上下文学习范式可行性，包括开放域编辑。代码已开源。
原文摘要 · Abstract (English)
Replicating In-Context Learning (ICL) in computer vision remains challenging due to task heterogeneity. We propose \textbf{VIRAL}, a framework that elicits visual reasoning from a pre-trained image editing model by formulating ICL as conditional generation via visual analogy ($x_s : x_t :: x_q : y_q$). We adapt a frozen Diffusion Transformer (DiT) using role-aware multi-image conditioning and introduce a Mixture-of-Experts LoRA to mitigate gradient interference across diverse tasks. Additionally, to bridge the gaps in current visual context datasets, we curate a large-scale dataset spanning perception, restoration, and editing. Experiments demonstrate that VIRAL outperforms existing methods, validating that a unified V-ICL paradigm can handle the majority of visual tasks, including open-domain editing. Our code is available at https://anonymous.4open.science/r/VIRAL-744A
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。