arXiv:2604.02073cs.CV2026-04被引 7

用隐式推理替代文字思维链,实现更快更准的多模态检索。

PLUME: Latent Reasoning Based Universal Multimodal Embedding

论文配图:PLUME: Latent Reasoning Based Universal Multimodal Embedding
图 1 · 摘自论文原文
  • 用连续潜变量的自回归演化代替文字思维链,降低计算开销。
  • 在MMEB-v2上达30倍以上推理加速,仅需不到10步潜变量更新。
  • 适合视频、图文文档等复杂结构数据的高效检索任务。

通用多模态嵌入(UME)将异构输入映射到共享检索空间,统一模型处理多种模态。现有方法通过生成显式的思维链(CoT)推理来提升复杂查询意图理解,但带来巨大推理开销,且易将丰富的多模态证据压缩至狭窄文本瓶颈。本文提出PLUME,一种基于隐式推理的框架,以短序列连续潜变量的自回归演化替代显式话语化思维链。为支持多样化多模态查询,PLUME引入语义锚引导的转换适配器,在固定计算预算下引导潜变量沿不同推理路径演化。为稳定训练,采用渐进式从显式到隐式的教学课程:训练初期使用显式推理作为支架,逐步转移至隐藏状态计算,推理阶段完全去除显式思维链。在78项任务的MMEB-v2基准测试中,PLUME超越强基线,将推理生成的词数从数百降至不足10个潜变量步骤,实现超30倍加速。该方法特别适用于证据密集、结构复杂且难以用语言组织的场景,如视频与视觉文档检索。结果表明,结构化潜变量计算可在无显式推理开销的前提下保留中间推理优势,为实际检索系统提供更强更高效的范式。

原文摘要 · Abstract (English)

Universal multimodal embedding (UME) maps heterogeneous inputs into a shared retrieval space with a single model. Recent approaches improve UME by generating explicit chain-of-thought (CoT) rationales before extracting embeddings, enabling multimodal large language models to better infer complex query intent. However, explicit CoT incurs substantial inference overhead and can compress rich multimodal evidence into a narrow textual bottleneck. We propose PLUME, a latent reasoning framework that advances UME by replacing verbalized CoT with a short autoregressive rollout of continuous latent states. To support diverse multimodal queries, PLUME further introduces a semantic-anchor-guided transition adapter that steers latent rollout along different reasoning trajectories under the same fixed computation budget. To stabilize training, PLUME adopts a progressive explicit-to-latent curriculum that uses verbalized reasoning only as a temporary training scaffold and gradually transfers this behavior into hidden-state computation, eliminating explicit CoT at inference. On the 78-task MMEB-v2 benchmark, PLUME outperforms strong explicit-CoT UME baselines while reducing reasoning from hundreds of generated tokens to fewer than 10 latent steps, delivering over 30x faster inference. PLUME is especially well suited to retrieval settings where relevant evidence is dense, structurally complex, and difficult to organize through verbalized intermediate rationales, such as video and visual document retrieval. These results show that structured latent computation can preserve the benefits of intermediate reasoning without the overhead of explicit rationale generation, providing a stronger and more efficient paradigm for practical retrieval systems.

多模态嵌入隐式推理检索加速潜变量

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。