arXiv:2606.13061cs.CV2026-06中稿 · EMNLP被引 1

让嵌入在潜在空间中思考,60倍提速且不依赖标注推理。

LaME: Learning to Think in Latent Space for Multimodal Embedding via Information Bottleneck

论文配图:LaME: Learning to Think in Latent Space for Multimodal Embedding via Information Bottleneck
图 1 · 摘自论文原文
  • 用可学习的潜变量令牌构建信息瓶颈,单次前向完成推理。
  • 在两个数据集上超越部分显式推理模型,推理速度提升60倍。
  • 无需文本推理标注,适合大规模多模态嵌入应用。

基于思维链(CoT)的通用多模态嵌入虽性能优越,但存在两大瓶颈:(i)自回归式CoT推理计算开销大,难以满足低延迟检索需求;(ii)嵌入性能高度依赖CoT标注质量,限制了大规模训练的可靠性。本文提出LaME(Latent Reasoning Multimodal Embedding),将面向嵌入的潜空间推理建模为弱监督信息瓶颈。LaME采用K个可学习的推理令牌作为固定容量瓶颈,所有推理在单次前向传播中完成。通过两种弱监督信号结构解耦对比与自回归目标,消除对CoT标注的依赖,结合两阶段训练策略保障收敛稳定性。在MMEB-v2和MRMR数据集上的实验表明,LaME达到竞争性性能,超越部分显式CoT模型,推理速度比显式CoT方法快60倍,比潜空间基线快2倍,吞吐量接近判别式嵌入模型。代码已开源。

原文摘要 · Abstract (English)

Reasoning-driven universal multimodal embedding has advanced rapidly by introducing Chain-of-Thought (CoT) reasoning into the embedding pipeline. Despite the strong performance across both general and complex tasks, this paradigm suffers from two core limitations: (i) autoregressive CoT reasoning incurs high computational cost, making it impractical for low-latency retrieval; and (ii) embedding performance is heavily coupled with CoT annotation quality, making large-scale training unreliable. These raise fundamental questions: Is textual CoT the optimal form of reasoning for embedding, and can effective embedding reasoning be accomplished in latent space? To this end, we propose LaME (Latent Reasoning Multimodal Embedding), which formulates embedding-oriented latent reasoning as a weakly supervised information bottleneck. LaME employs K learnable reason tokens as a fixed-capacity bottleneck, completing all reasoning within a single forward pass. The two weak supervision signals structurally decouple contrastive from autoregressive objectives and eliminate dependence on CoT annotations, while a two-stage training pipeline ensures stable convergence. Experiments on MMEB-v2 and MRMR show that LaME achieves competitive performance, surpassing some explicit CoT-based models, while delivering 60x faster inference than explicit CoT methods and 2x faster than latent baselines with throughput comparable to discriminative embedding models. Code is available at https://github.com/PeppaWu/LaME.

多模态嵌入潜空间推理信息瓶颈高效推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。