用重写机制替代思维链，提升多模态生成嵌入的效率与准确率
Beyond Chain-of-Thought: Rewrite as a Universal Interface for Generative Multimodal Embeddings
- 以重写任务统一生成与嵌入优化，减少冗余推理步骤
- 在MMEB-V2等数据集上优于现有生成式嵌入模型，思维长度显著缩短
- 适合需要高效精准跨模态检索的研究者与开发者
多模态大语言模型（MLLM）已成为通用多模态嵌入的有力基础。近期研究发现，基于推理的生成式嵌入在多个嵌入任务中优于判别式嵌入。然而，思维链（CoT）推理在更广泛的检索场景中常产生冗余思考步骤，并导致摘要答案语义模糊。为此，我们提出重写驱动的多模态嵌入（RIME）框架，通过检索友好的重写联合优化生成与嵌入。同时引入跨模态对齐（CMA），连接生成式与判别式嵌入空间，实现灵活的双向检索，平衡效率与精度。进一步提出精炼强化学习（Refine-RL），将判别式嵌入作为稳定的语义锚点，指导重写优化。在MMEB-V2、MRMR和UVRB上的大量实验表明，RIME显著超越现有生成式嵌入模型，且思维长度大幅缩减。代码已开源：https://github.com/PeppaWu/RIME。
原文摘要 · Abstract (English)
Multimodal Large Language Models (MLLMs) have emerged as a promising foundation for universal multimodal embeddings. Recent studies have shown that reasoning-driven generative multimodal embeddings can outperform discriminative embeddings on several embedding tasks. However, Chain-of-Thought (CoT) reasoning tends to generate redundant thinking steps and introduce semantic ambiguity in the summarized answers in broader retrieval scenarios. To address this limitation, we propose Rewrite-driven Multimodal Embedding (RIME), a unified framework that jointly optimizes generation and embedding through a retrieval-friendly rewrite. Meanwhile, we present the Cross-Mode Alignment (CMA) to bridge the generative and discriminative embedding spaces, enabling flexible mutual retrieval to trade off efficiency and accuracy. Based on this, we also introduce Refine Reinforcement Learning (Refine-RL) that treats discriminative embeddings as stable semantic anchors to guide the rewrite optimization. Extensive experiments on MMEB-V2, MRMR and UVRB demonstrate that RIME substantially outperforms prior generative embedding models while significantly reducing the length of thinking. Code is available at https://github.com/PeppaWu/RIME.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。