谷歌嵌入模型在多语言检索中表现最佳，但速度慢，本地部署可选mE5-L。
Benchmarking Google Embeddings 2 against Open-Source Models for Multilingual Dense Retrieval and RAG Systems
- 对比六种模型，谷歌嵌入2.0在多语言密集检索中全面领先
- 在意大利语RAG任务上达到nDCG@10=0.282，但延迟高达231.6毫秒
- 若需低于100毫秒响应，mE5-L是更优的本地替代方案
我们对谷歌嵌入2.0（GE2）——一种支持2048词元上下文且具备显式任务类型条件的双编码器——与五种开源模型（BGE-M3、E5-large、Multilingual-E5-large、LaBSE、Paraphrase-Multilingual-MPNet）进行了基准测试。评估涵盖四个BEIR子集、一个合成意大利语RAG数据集、三种分块策略下五种词元大小的消融实验，以及在普通CPU上的每查询延迟。GE2在所有任务中排名第一，平均nDCG@10达0.638，意大利语RAG任务nDCG@10为0.282，但中位延迟为231.6毫秒，约为最快本地模型的14倍。mE5-L在意大利语任务上仅落后0.003 nDCG，延迟仅31毫秒，适合低延迟场景。有趣的是，尽管广泛用于多语言任务，LaBSE在BEIR上的平均nDCG@10仅为0.188，低于所有专用检索模型，包括mMPNet。分块实验表明，所有六种模型在32词元分块时已饱和，仅在16词元时语义分块带来显著提升。
原文摘要 · Abstract (English)
We benchmark Google Embeddings (GE2), a Vertex-AI-hosted bi-encoder with 2,048-token context and explicit task-type conditioning, against five open-source alternatives: BGE-M3, E5-large, Multilingual-E5-large (mE5-L), LaBSE, and Paraphrase-Multilingual-MPNet (mMPNet). Evaluation covers four BEIR subsets, a synthetic Italian RAG corpus, a chunking ablation considering 5 sizes of tokens with three strategies, and per-query latency on commodity CPU hardware. GE2 ranks first on every task, achieving BEIR avg.nDCG@10 = 0.638 and IT-RAG-Bench nDCG@10 = 0.282, but at 231.6 ms median latency, it is roughly 14x slower than the fastest local models. mE5-L reaches within 0.003 nDCG of GE2 on Italian at 31 ms, making it the preferred option when sub-100 ms SLAs matter. A more striking finding concerns LaBSE, which, despite widespread multilingual deployment scores 0.188 average nDCG@10 on BEIR, below every dedicated retrieval model including mMPNet. Chunking experiments show that all six models saturate at 32-token chunks on our corpus, with semantic chunking providing measurable gains only at 16 tokens.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。