arXiv:2608.24053cs.CVcs.CL2026-08

微信多模态嵌入模型,支持图文视频混合输入,90亿参数达顶尖性能。

WeMM-Embedding: WeChat Multi-Modal Embedding Technical Report

论文配图:WeMM-Embedding: WeChat Multi-Modal Embedding Technical Report
图 1 · 摘自论文原文
  • 分两阶段训练:大规模对齐+精细化优化,支持多种模态混合输入。
  • 90亿参数版本在多个评测中达80.6分新高,20亿参数版已超此前最佳开源模型。
  • 已在微信搜索、推荐等14个线上场景部署,显著提升效果。

通用多模态嵌入正成为现代AI系统的核心组件,可将异构内容映射到共享空间,用于检索、推荐、分类及智能体系统。本文介绍WeMM-Embedding,一个支持文本、图像、视频、视觉文档及任意交错多模态输入的通用多模态嵌入模型家族,包含20亿、40亿和90亿参数版本。模型采用两阶段训练:先进行大规模多模态对齐,再通过精选数据、细粒度相关性监督与跨尺度知识迁移进行精炼。在多项公开基准测试中,WeMM-Embedding表现领先。其中,20亿参数版本已超越此前领先的80亿开源基线(MMEB-v2),90亿版本更取得80.6的新最佳综合得分。该模型在微信内部26项任务的评测中表现出色,14项线上A/B测试均实现持续提升,并已大规模部署于微信频道、公众号、朋友圈及电商服务等场景。模型权重与代码已开源,详见https://github.com/Tencent/WeMM-Embedding。

原文摘要 · Abstract (English)

Universal multimodal embeddings are becoming a core component of modern AI systems, enabling heterogeneous content to be represented in a shared space for applications such as retrieval, recommendation, classification, and agentic systems. In this report, we present WeMM-Embedding, a family of universal multimodal embedding models supporting text, images, videos, visual documents, and arbitrarily interleaved multimodal inputs with flexible output dimensions. The family comprises 2B, 4B, and 9B variants and is trained in two stages: a large-scale multimodal alignment stage, followed by a refinement stage using curated data, fine-grained relevance supervision, and cross-scale knowledge transfer. Across extensive evaluations, WeMM-Embedding achieves leading performance on multiple public benchmarks. Notably, the 2B variant already surpasses the previously leading 8B open-source baseline on MMEB-v2, while the 9B variant further achieves a new state-of-the-art overall score of 80.6. WeMM-Embedding also demonstrates strong practical performance across WeChat applications, with substantial gains on a 26-task in-house benchmark and consistent improvements across 14 online A/B tests. It has been deployed at scale across recommendation and search applications, including WeChat Channels, Official Accounts, Moments, and e-commerce services. We have released the model weights and code to facilitate future research at https://github.com/Tencent/WeMM-Embedding.

多模态嵌入模型微信开源

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。