通过成对训练提升多模态模型的理解与生成能力
PairUni: Pairwise Training for Unified Multimodal Language Models
- 将视觉与语言任务配对，构建跨模态语义对齐数据
- 在1B到14B规模模型上均实现性能提升，图像编辑任务零样本改进
- 适合追求统一架构下多任务协同优化的研究者
统一多模态语言模型（UVLMs）在单一架构中同时完成理解与生成。由于依赖异构数据与监督信号，强化学习中平衡生成与理解极具挑战。为此，我们提出PairUni，一种将数据重组为理解-生成（UG）配对并相应对齐优化的统一框架。具体地，通过跨模态语义补全合成对齐实例，并检索语义相关样本，构建统一配对数据集。这些配对结构揭示了跨任务语义对应关系，支持一致策略学习。为利用该结构，我们提出基于组相对策略优化（PairGRPO）的配对感知变体，为每对分配相似性得分以调节优势值，增强对齐良好样本的学习，减少任务干扰。在多种UVLM架构（自回归与离散扩散）及规模（1B至14B）上的广泛实验表明，PairUni持续优于强基线。值得注意的是，该方法在未使用任何编辑特定数据的情况下，仍显著提升图像编辑任务表现。代码已开源。
原文摘要 · Abstract (English)
Unified Vision-Language Models (UVLMs) perform both understanding and generation within a single architecture. Since these models rely on heterogeneous data and supervision, balancing both generation and understanding in reinforcement learning (RL) is challenging. To address this challenge, we propose PairUni, a unified framework that reorganizes data into understanding-generation (UG) pairs and aligns optimization accordingly. Specifically, we construct a unified paired dataset by synthesizing aligned instances via cross-modal semantic completion and retrieving semantically related samples. These paired structures expose cross-task semantic correspondences and support consistent policy learning. To leverage this structure, we present PairGRPO, a pair-aware variant based on Group Relative Policy Optimization. It assigns a similarity score to each pair to modulate the advantage, strengthening learning from well-aligned examples and reducing task interference. Extensive experiments across diverse UVLM architectures (Autoregressive and Discrete Diffusion) and scales (1B to 14B) demonstrate that PairUni yields consistent improvements over strong baselines. Notably, our method also demonstrates strong generalization by improving performance on image editing tasks without using any editing-specific data. Codes are available at https://github.com/Haochen-Wang409/PairUni.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。