用多跳推理数据训练视觉语言模型,显著提升复杂问答能力
HopChain: Multi-Hop Data Synthesis for Generalizable Vision-Language Reasoning
- 构建多跳推理数据生成框架,每步依赖视觉证据逐步推导
- 在24个基准上,20项性能提升,长链推理最多增50分
- 适合需要强逻辑推理的视觉理解任务,如科学问题与文档分析
视觉语言模型虽具强大多模态能力,但在细粒度视觉语言推理上仍存不足。我们发现长链思维(CoT)推理会暴露感知、推理、知识和幻觉等多重错误,且错误随步骤累积。现有用于可验证奖励强化学习(RLVR)的数据缺乏依赖视觉证据的复杂推理链,未能充分暴露这些缺陷。为此,我们提出HopChain,一个可扩展的多跳视觉语言推理数据合成框架。每个合成的多跳查询形成逻辑依赖的实例化步骤链,前期步骤建立后续所需实例、集合或条件,最终答案为具体明确数值,便于验证奖励。我们在Qwen3.5-35B-A3B和Qwen3.5-397B-A17B上进行两种RLVR设置对比:仅原始数据,以及原始数据+HopChain多跳数据,并在涵盖STEM与谜题、通用VQA、文本识别与文档理解、视频理解的24个基准上测试。尽管该多跳数据未针对特定基准设计,却在20个基准上均提升性能,表明广泛通用性。将完整链式查询替换为半链或多跳变体,五个代表性基准平均得分从70.4降至66.7和64.3。值得注意的是,在超长链推理中,多跳收益最高,超过50分。实验验证了HopChain作为有效、可扩展的多跳数据生成框架,能显著提升视觉语言模型的泛化推理能力。
原文摘要 · Abstract (English)
Vision-language models (VLMs) show strong multimodal capabilities but still struggle with fine-grained vision-language reasoning. We find that long chain-of-thought (CoT) reasoning exposes diverse failure modes, including perception, reasoning, knowledge, and hallucination errors, which can compound across intermediate steps. However, most existing vision-language data used for reinforcement learning with verifiable rewards (RLVR) does not involve complex reasoning chains that rely on visual evidence throughout, leaving these weaknesses largely unexposed. We therefore propose HopChain, a scalable framework for synthesizing multi-hop vision-language reasoning data for RLVR training of VLMs. Each synthesized multi-hop query forms a logically dependent chain of instance-grounded hops, where earlier hops establish the instances, sets, or conditions needed for later hops, while the final answer remains a specific, unambiguous number suitable for verifiable rewards. We train Qwen3.5-35B-A3B and Qwen3.5-397B-A17B under two RLVR settings: the original data alone, and the original data plus HopChain's multi-hop data, and compare them across 24 benchmarks spanning STEM and Puzzle, General VQA, Text Recognition and Document Understanding, and Video Understanding. Although this multi-hop data is not synthesized for any specific benchmark, it improves 20 of 24 benchmarks on both models, indicating broad and generalizable gains. Consistently, replacing full chained queries with half-multi-hop or single-hop variants reduces the average score across five representative benchmarks from 70.4 to 66.7 and 64.3, respectively. Notably, multi-hop gains peak in long-CoT vision-language reasoning, exceeding 50 points in the ultra-long-CoT regime. These experiments establish HopChain as an effective, scalable framework for synthesizing multi-hop data that improves generalizable vision-language reasoning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。