让视觉语言模型同时用文字和连续图像特征推理,提升复杂任务表现
HyLaR: Hybrid Latent Reasoning with Decoupled Policy Optimization

- 将文本生成与连续视觉隐变量混合推理,打破离散化瓶颈
- 在多个基准上超越现有方法,细粒度感知任务提升12.3%
- 适合需要精准视觉理解的多模态应用,如医疗影像分析
思维链(CoT)推理显著提升了多模态大语言模型(MLLMs)解决复杂问题的能力。然而,将CoT应用于视觉时通常需将信号离散化以适配语言模型输入,导致早期语义坍缩并丢失细粒度信息。尽管外部工具可缓解此问题,但引入刚性瓶颈,使推理受限于预定义操作。虽有近期隐变量推理范式将视觉状态内化以克服上述限制,但优化由此产生的混合离散-连续动作空间仍具挑战。本文提出HyLaR(混合隐变量推理)框架,无缝交织离散文本生成与连续视觉隐表示。具体而言,在初始冷启动监督微调(SFT)后,引入DePO(解耦策略优化)以实现该混合空间内的有效强化学习。DePO分解策略梯度目标,对文本与隐变量组件分别施加独立的信任域约束,并采用精确闭式冯·米塞斯-费舍尔(vMF)KL正则项。大量实验表明,HyLaR在细粒度感知与通用多模态理解基准上均优于标准MLLM及最先进的隐变量推理方法。代码已公开于https://github.com/EthenCheng/HyLaR。
原文摘要 · Abstract (English)
Chain-of-Thought (CoT) reasoning significantly elevates the complex problem-solving capabilities of multimodal large language models (MLLMs). However, adapting CoT to vision typically discretizes signals to fit LLM inputs, causing early semantic collapse and discarding fine-grained details. While external tools can mitigate this, they introduce a rigid bottleneck, confining reasoning to predefined operations. Although recent latent reasoning paradigms internalize visual states to overcome these limitations, optimizing the resulting hybrid discrete-continuous action space remains challenging. In this work, we propose HyLaR (Hybrid Latent Reasoning), a framework that seamlessly interleaves discrete text generation with continuous visual latent representations. Specifically, following an initial cold-start supervised fine-tuning (SFT), we introduce DePO (Decoupled Policy Optimization) to enable effective reinforcement learning within this hybrid space. DePO decomposes the policy gradient objective, applying independent trust-region constraints to the textual and latent components, alongside an exact closed-form von Mises-Fisher (vMF) KL regularizer. Extensive experiments demonstrate that HyLaR outperforms standard MLLMs and state-of-the-art latent reasoning approaches across fine-grained perception and general multimodal understanding benchmarks. Code is available at https://github.com/EthenCheng/HyLaR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。