解决多模态模型推理中的答案泄露问题,提升连续空间推理准确性。
Multimodal Continuous Reasoning via Asymmetric Mutual Variational Learning

- 采用双向校准的变分学习框架,分离训练与推理路径。
- 在BLINK基准上平均得分提升10.83,个别任务最高增32.00。
- 适合追求高精度多模态推理的科研与工业应用。
多模态大语言模型常受限于语言空间瓶颈,将复杂视觉推理压缩为离散标记,导致感知细节丢失。连续潜在推理可通过隐式推理路径连接多模态查询与最终答案,但引入严重训练-推理不匹配:训练时的后验依赖真实答案,可能利用答案相关捷径。标准变分训练迫使推理前验模仿一个包含测试时不可用信息的后验,导致性能下降。为此,我们提出非对称互变分学习(AMVL),通过双向校准目标解决此问题。前向KL损失使目标无关前验匹配后验,而新颖的反向KL损失同时正则化后验,防止其坍缩至推理不兼容区域,缓解‘答案泄露’。理论分析将此泄露形式化为前验污染,并证明双KL目标可降低该现象。我们在潜变量融合的MLLM中实现AMVL,结果持续优于强基线,在复杂BLINK基准上平均得分提升10.83,单任务最高提升32.00,分析验证了潜空间稳定性提升。
原文摘要 · Abstract (English)
Multimodal Large Language Models (MLLMs) are often constrained by a language-space bottleneck, forcing complex visual reasoning into discrete tokens which can lose perceptual nuance. A promising alternative is continuous latent reasoning, where the goal is to discover implicit reasoning pathways that bridge the multimodal query and the final answer. However, this introduces a severe train-inference mismatch: a training-time posterior, conditioned on the ground-truth answer, can exploit answer-dependent shortcuts. Standard variational training then forces the inference-time prior to mimic a posterior that has access to information unavailable at test time, leading to poor performance. To address this, we propose Asymmetric Mutual Variational Learning (AMVL), a framework that resolves this mismatch via a bidirectional calibration objective. A forward KL divergence trains the target-agnostic prior to match the posterior, while a novel reverse KL divergence simultaneously regularizes the posterior, preventing it from collapsing into inference-incompatible regions and mitigating this ``answer leakage''. We provide theoretical analysis formalizing this leakage as prior contamination and prove that our dual-KL objective reduces it. We instantiate AMVL in a latent-integrated MLLM and show that it consistently outperforms strong discrete and latent-reasoning baselines, improving the average score on the complex BLINK benchmark by +10.83 and achieving gains of up to +32.00 on individual reasoning tasks, with analyses confirming improved latent-space stability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。