arXiv:2508.03320cs.CV2025-08被引 18

15亿参数模型统一图文理解生成,能在普通显卡上高效运行。

Skywork UniPic: Unified Autoregressive Modeling for Visual Understanding and Generation

  • 分离编码:用掩码自回归编码器合成,SigLIP2编码器理解,共享解码器
  • 生成1024×1024图像仅需少于15GB显存,复杂生成任务达85.5新纪录
  • 适合部署在消费级硬件的高保真多模态应用,如图像编辑与生成

我们提出Skywork UniPic,一个15亿参数的自回归模型,将图像理解、文生图和图像编辑统一于单一架构中,无需特定任务适配器或模块连接;在消费级硬件上实现顶尖性能。该模型在GenEval上得分0.86,超越多数统一模型;在DPG-Bench复杂生成任务中创85.5新纪录;图像编辑任务上获GEditBench-EN 5.83分和ImgEdit-Bench 3.49分;支持1024×1024图像生成,且显存占用低于15GB(如RTX 4090)。其核心设计包括:(1) 解耦编码策略——使用掩码自回归编码器用于生成,SigLIP2编码器用于理解,两者共同驱动共享自回归解码器;(2) 逐步分辨率训练策略,从256×256到1024×1024动态解冻参数,平衡能力与稳定性;(3) 经过精心构建的1亿规模数据集,结合任务专属奖励模型优化生成与编辑目标。结果表明,高质量多模态集成无需高昂资源开销,为可部署的高保真多模态AI提供可行路径。代码与权重已公开于https://huggingface.co/Skywork/Skywork-UniPic-1.5B。

原文摘要 · Abstract (English)

We introduce Skywork UniPic, a 1.5 billion-parameter autoregressive model that unifies image understanding, text-to-image generation, and image editing within a single architecture-eliminating the need for task-specific adapters or inter-module connectors-and demonstrate that compact multimodal systems can achieve state-of-the-art performance on commodity hardware. Skywork UniPic achieves a GenEval score of 0.86, surpassing most existing unified models; sets a new DPG-Bench complex-generation record of 85.5; attains 5.83 on GEditBench-EN and 3.49 on ImgEdit-Bench for image editing; and generates 1024 x 1024 images with under 15 GB of GPU memory (e.g., RTX 4090). (1) a decoupled encoding strategy that leverages a masked autoregressive encoder for synthesis and a SigLIP2 encoder for understanding, all feeding a shared autoregressive decoder; (2) a progressive, resolution-aware training schedule scaling from 256 x 256 to 1024 x 1024 while dynamically unfreezing parameters to balance capacity and stability; and (3) meticulously curated, 100 million-scale datasets augmented with task-specific reward models to refine generation and editing objectives. By demonstrating that high-fidelity multimodal integration need not incur prohibitive resource demands, Skywork UniPic establishes a practical paradigm for deployable, high-fidelity multimodal AI. Code and weights are publicly available at https://huggingface.co/Skywork/Skywork-UniPic-1.5B.

多模态图像生成自回归轻量化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。