让所有视觉信息都用RGB表示，统一视觉任务为图像编辑问题。
Let RGB Be the Language of Vision
- 将掩码、深度图等视觉信号统一编码为RGB，实现跨任务共享模型结构。
- 零样本下在分割、深度估计等任务上表现良好，输入输出均为RGB格式。
- 适合研究通用视觉系统或希望简化多模态接口的开发者参考。
本文提出一种统一的视觉模型框架，将超出自然图像的各类视觉信息（如掩码、深度图等）均表示为RGB图像，并将各种视觉任务转化为通用的RGB到RGB图像编辑问题。在此范式下，不同类型的视觉信息内部共享相同的编码与解码架构及参数，使单一模型可通过统一的视觉接口实现任务间迁移，类似语言模型处理文本的方式。该框架称为RGB输入-输出（RINO）。基于一个通用图像编辑主干网络，无需特定任务微调，RINO在密集理解任务（如分割、深度估计，输出统一为RGB）和密集条件生成任务（如姿态转图像，输入统一为RGB）中均展现出稳健且具有竞争力的零样本性能。本研究旨在为通用统一的视觉-语言系统提供启示，使多样化的视觉任务可通过共享的视觉语言表达、解释与求解。代码已开源：https://github.com/yangtiming/RINO。
原文摘要 · Abstract (English)
This work introduces a unified formulation for vision models, where diverse forms of visual information beyond natural images, such as masks, depth maps, and other structured visual signals, are all represented as RGB images, while general visual tasks can be converted into a common RGB-to-RGB image editing problem. In this paradigm, different types of visual information internally share the same encoding and decoding architecture and parameters as natural images, enabling a single model to transfer across tasks through a unified visual interface, in a way analogous to how language models operate over text. We refer to this formulation as RGB In and RGB Out (RINO). Built upon a generic image editing backbone without task-specific fine-tuning, RINO demonstrates robust and competitive zero-shot performance on both dense understanding tasks such as segmentation and depth estimation (where we unify outputs as RGB), and dense-conditioned generation tasks such as pose-to-image generation (where we unify inputs as RGB). We hope this study provides useful insights toward general unified vision-language systems, where diverse visual tasks can be expressed, interpreted, and solved through a shared visual language. Code is available at https://github.com/yangtiming/RINO.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。