用文本推理数据微调模型,让多模态模型学会慢思考。
Virgo: A Preliminary Exploration on Reproducing o1-like MLLM
- 用少量长文本推理数据微调MLLM,实现多模态慢思考。
- 纯文本推理数据比视觉推理数据更有效激发慢思考能力。
- 证明慢思考能力主要来自语言模型组件,可跨模态迁移。
近期,基于大语言模型(LLM)的慢思考推理系统因在推理时延长思考时间而受到广泛关注。将此能力拓展至多模态大语言模型(MLLM)也日益成为研究热点。由于MLLM需处理跨模态的复杂语义,实现多模态慢思考更具挑战性。本文提出一种简单方法:仅用少量文本形式的长序列推理数据对强大的MLLM进行微调,构建出名为Virgo(Visual reasoning with long thought)的多模态慢思考系统。实验发现,自然语言表达的长序列推理过程可有效迁移到MLLM中。更重要的是,此类文本推理数据似乎比视觉推理数据更能激发MLLM的慢思考能力。尽管本工作为初步探索,但表明慢思考能力本质上与语言模型组件相关,可跨模态或领域迁移。该发现有助于指导更强大慢思考推理系统的开发。代码与资源已开源:https://github.com/RUCAIBox/Virgo。
原文摘要 · Abstract (English)
Recently, slow-thinking reasoning systems, built upon large language models (LLMs), have garnered widespread attention by scaling the thinking time during inference. There is also growing interest in adapting this capability to multimodal large language models (MLLMs). Given that MLLMs handle more complex data semantics across different modalities, it is intuitively more challenging to implement multimodal slow-thinking systems. To address this issue, in this paper, we explore a straightforward approach by fine-tuning a capable MLLM with a small amount of textual long-form thought data, resulting in a multimodal slow-thinking system, Virgo (Visual reasoning with long thought). We find that these long-form reasoning processes, expressed in natural language, can be effectively transferred to MLLMs. Moreover, it seems that such textual reasoning data can be even more effective than visual reasoning data in eliciting the slow-thinking capacities of MLLMs. While this work is preliminary, it demonstrates that slow-thinking capacities are fundamentally associated with the language model component, which can be transferred across modalities or domains. This finding can be leveraged to guide the development of more powerful slow-thinking reasoning systems. We release our resources at https://github.com/RUCAIBox/Virgo.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。