用多模态大模型做图像生成评分,更准更快。
Multimodal LLMs as Customized Reward Models for Text-to-Image Generation
- 直接用多模态模型的隐藏状态评估图文生成质量。
- 引入跨层连接模块,提升视觉与文本关联推理能力。
- 支持多种偏好数据,适合高效训练和生成优化。
我们提出一种名为LLaVA-Reward的高效奖励模型,用于自动评估文本到图像(T2I)生成结果的多个维度。现有基于多模态大语言模型(MLLM)的方法依赖指令跟随数据进行有监督微调,并通过分析文本响应来评估生成质量,耗时且难训练。为此,我们设计了直接利用给定图文对的多模态大模型隐藏状态的方案。为增强解码器仅架构下视觉与文本表征的双向交互,我们进一步提出跳连交叉注意力(SkipCA)模块,通过将早期视觉特征与后期隐藏表示相连,提升图文相关性推理能力。此外,LLaVA-Reward支持多种偏好数据类型,包括成对和非成对数据,实现高效微调。我们在四个评价维度上训练该模型:图文对齐性、保真度/伪影、安全性及整体排名。实验证明,相比传统方法和基于MLLM的方法,LLaVA-Reward在生成符合人类偏好的评分以及生成阶段缩放方面表现更优。
原文摘要 · Abstract (English)
We introduce LLaVA-Reward, an efficient reward model designed to automatically evaluate text-to-image (T2I) generations across multiple perspectives, leveraging pretrained multimodal large language models (MLLMs). Existing MLLM-based approaches require instruction-following data for supervised fine-tuning and evaluate generation quality on analyzing text response, which is time-consuming and difficult to train. To address this problem, we propose LLaVA-Reward, which directly utilizes the hidden states of MLLMs given text-image pairs. To enhance the bidirectional interaction between visual and textual representations in decoder-only MLLMs, we further propose adding a Skip-connection Cross Attention (SkipCA) module. This design enhances text-image correlation reasoning by connecting early-layer visual features with later-layer hidden representations. In addition, LLaVA-Reward supports different types of preference data for efficient fine-tuning, including paired preference data and unpaired data. We train LLaVA-Reward on four evaluation perspectives: text-image alignment, fidelity/artifact, safety, and overall ranking. Empirical results demonstrate that LLaVA-Reward outperforms conventional and MLLM-based methods in generating human-aligned scores for automatic evaluations and inference-time scaling in text-to-image generations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。