VARGPT-v1.1通过迭代指令微调与强化学习,提升多模态理解与图像生成能力。
VARGPT-v1.1: Improve Visual Autoregressive Large Unified Model via Iterative Instruction Tuning and Reinforcement Learning
- 结合迭代指令微调与直接偏好优化(DPO)进行训练
- 在830万图文指令对上训练,支持高分辨率图像生成
- 无需修改架构即可实现图像编辑,适合多任务应用
本文提出VARGPT-v1.1,一种先进的统一视觉自回归模型,在前序框架基础上进行升级。模型保持下一标记预测(用于视觉理解)与下一尺度生成(用于图像合成)的双范式。具体改进包括:(1) 采用结合迭代视觉指令微调与基于直接偏好优化(DPO)的强化学习的新训练策略;(2) 扩展训练语料库至包含830万条视觉-生成指令对;(3) 升级语言模型主干为Qwen2;(4) 提升图像生成分辨率;(5) 实现无需架构修改的图像编辑能力。这些改进使VARGPT-v1.1在多模态理解与文本到图像指令遵循任务中达到当前最优性能,显著提升理解和生成指标。通过视觉指令微调,模型获得图像编辑功能,同时保持与前代一致的架构,揭示统一视觉模型在理解、生成与编辑方面的潜力。研究表明,精心设计的统一视觉自回归模型可有效借鉴大语言模型的灵活训练策略,具备良好可扩展性。代码与模型权重已公开于https://github.com/VARGPT-family/VARGPT-v1.1。
原文摘要 · Abstract (English)
In this work, we present VARGPT-v1.1, an advanced unified visual autoregressive model that builds upon our previous framework VARGPT. The model preserves the dual paradigm of next-token prediction for visual understanding and next-scale generation for image synthesis. Specifically, VARGPT-v1.1 integrates: (1) a novel training strategy combining iterative visual instruction tuning with reinforcement learning through Direct Preference Optimization (DPO), (2) an expanded training corpus containing 8.3M visual-generative instruction pairs, (3) an upgraded language model backbone using Qwen2, (4) enhanced image generation resolution, and (5) emergent image editing capabilities without architectural modifications. These advancements enable VARGPT-v1.1 to achieve state-of-the-art performance in multimodal understanding and text-to-image instruction-following tasks, demonstrating significant improvements in both comprehension and generation metrics. Notably, through visual instruction tuning, the model acquires image editing functionality while maintaining architectural consistency with its predecessor, revealing the potential for unified visual understanding, generation, and editing. Our findings suggest that well-designed unified visual autoregressive models can effectively adopt flexible training strategies from large language models (LLMs), exhibiting promising scalability. The codebase and model weights are publicly available at https://github.com/VARGPT-family/VARGPT-v1.1.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。