统一加载多类视觉语言模型,让不同架构可复现对比。
UVLM: A Universal Vision-Language Model Loader for Reproducible Multimodal Benchmarking
- 用统一接口抽象不同模型的差异,支持快速切换和测试。
- 在120张街景图上验证,模型在复杂推理任务中表现差距明显。
- 支持自定义提示与思维链模式,适合做公平对比实验的研究者。
视觉语言模型(VLMs)在图像理解任务中表现强大,但因架构差异大,实际部署困难。本文提出UVLM(通用视觉语言模型加载器),一个基于Google Colab的框架,提供统一接口用于加载、配置和评估多种VLM架构在自定义图像分析任务上的表现。当前支持LLaVA-NeXT和Qwen2.5-VL两大模型家族,它们在视觉编码、分词和解码策略上存在根本差异。该框架通过单一推理函数屏蔽这些差异,使研究者能使用相同提示和评估协议进行模型对比。关键功能包括:支持数值、类别、布尔、文本四种输出类型的多任务提示生成器;基于多次推理结果多数投票的共识验证机制;灵活的1,500令牌预算,支持通过提示工程设计自定义推理策略;以及内置的思维链参考模式用于基准测试。UVLM注重可复现性、可访问性和可扩展性,可在消费级GPU资源的Google Colab上免费部署。论文首次基于120张街景图像的语料库,对不同VLM在逐步增加推理复杂度的任务上进行了基准测试。
原文摘要 · Abstract (English)
Vision-Language Models (VLMs) have emerged as powerful tools for image understanding tasks, yet their practical deployment remains hindered by significant architectural heterogeneity across model families. This paper introduces UVLM (Universal Vision-Language Model Loader), a Google Colab-based framework that provides a unified interface for loading, configuring, and benchmarking multiple VLM architectures on custom image analysis tasks. UVLM currently supports two major model families -- LLaVA-NeXT and Qwen2.5-VL -- which differ fundamentally in their vision encoding, tokenization, and decoding strategies. The framework abstracts these differences behind a single inference function, enabling researchers to compare models using identical prompts and evaluation protocols. Key features include a multi-task prompt builder with support for four response types (numeric, category, boolean, text), a consensus validation mechanism based on majority voting across repeated inferences, a flexible token budget (up to 1,500 tokens) enabling users to design custom reasoning strategies through prompt engineering, and a built-in chain-of-thought reference mode for benchmarking. UVLM is designed for reproducibility, accessibility, and extensibility and as such is freely deployable on Google Colab using consumer-grade GPU resources. The paper also presents the first benchmarking of different VLMs on tasks of increasing reasoning complexity using a corpus of 120 street-view images.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。