arXiv:2607.28751cs.CV2026-07

通过可学习的循环检索寄存器,实现高效通用多模态嵌入。

ReLoop-UME: Recurrent Depth with Learnable Retrieval Registers for Universal Multimodal Embedding

论文配图:ReLoop-UME: Recurrent Depth with Learnable Retrieval Registers for Universal Multimodal Embedding
图 1 · 摘自论文原文
  • 采用循环重用检索块,固定令牌空间扩展计算深度。
  • 在MMEB-V2和MRMR上提升检索性能,速度比UME-R1快44.9倍。
  • 适合需要低延迟高精度多模态检索的应用场景。

通用多模态嵌入(UME)将异构多模态输入映射到共享嵌入空间。现有方法或仅通过单次前向编码生成嵌入,或通过显式理由令牌和潜在自回归状态增加计算量。尽管令牌扩展可提升复杂匹配能力,但串行生成会增加检索延迟,并使最终嵌入依赖于生成的中间状态。本文提出新问题:能否在保持令牌工作区不变的前提下,通过模型深度扩展有用计算?我们分析独立训练的多个UME模型各层的正负样本相似度分离情况,发现共性规律:早期层对多模态输入进行语境化处理,中后期连续阶段形成具有检索判别性的特征,最终层将其映射为嵌入空间。基于此,提出ReLoop-UME:仅执行一次早期层,循环重用参数共享的检索形成块,并在最后一次循环后应用最终映射层。可学习的检索寄存器提供持续的检索特异性状态,跨循环累积与交换证据,最终寄存器作为嵌入读出。在MMEB-V2和MRMR数据集上,ReLoop-UME在不同主干网络下均一致提升检索性能,运行速度比UME-R1快44.9倍,比PLUME快1.5倍。

原文摘要 · Abstract (English)

Universal multimodal embedding (UME) maps heterogeneous multimodal inputs into a shared embedding space. Existing UME models either form embeddings through single forward encoding or add computation through explicit rationale tokens and latent autoregressive states. Although token expansion can improve complex matching, serial generation increases retrieval latency and makes the final embedding depend on generated intermediate states. This raises a different question: can useful computation be expanded along model depth while keeping the token workspace fixed? We analyze positive-negative similarity separation at every layer of independently trained UME models and observe a shared progression: early layers contextualize multimodal inputs, a contiguous middle-to-late stage forms retrieval-discriminative features, and the final layers map them into the embedding space. Based on this finding, we propose ReLoop-UME, which executes the early layers once, recurrently reuses a parameter-shared retrieval-forming block, and applies the final mapping layers after the last loop. Learnable Retrieval Registers provide persistent retrieval-specific states that accumulate and exchange evidence across loops, with the final register serving as the embedding readout. On MMEB-V2 and MRMR, ReLoop-UME consistently improves retrieval across different backbones while running 44.9x faster than UME-R1 and 1.5x faster than PLUME.

多模态嵌入循环结构检索优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。