测试静态嵌入对荷兰语混合检索的增益，发现其无法带来额外价值。
Do Static Embeddings Add Value to Hybrid Dutch Retrieval?
- 用加权倒数排名融合结合词法与模型检索，评估静态嵌入作用
- 四种任务中融合均提升效果，但静态嵌入始终未获正权重
- 结果表明基准性能不等于实际融合中的边际贡献，适合检索研究者
嵌入基准衡量独立模型质量，但无法判断在已有词法与Transformer检索基础上，低成本检索器是否提供互补排序信息。本文在荷兰语检索任务（MTEB-NL）上进行受控评估，采用加权倒数排名融合（RRF）组合最佳匹配25（BM25）、Qwen/Qwen3-Embedding-0.6B（Qwen）及两个多语言静态嵌入模型。共5个数据集，包含14,500个查询和786,573个文档，融合权重在单纯形上以0.1为步长搜索。十折查询级交叉验证在九折上选择权重，于留出折上评估；配对引导置信区间与符号随机化检验量化差异。融合在荷兰新闻、VABB、WebFAQ NL和Wikipedia NL任务上分别优于训练选定的单一检索器0.061、0.029、0.004和0.025的平均倒数秩（MRR），在Open Tender上与BM25持平。四个正向差异在霍尔姆校正后仍显著。所有50次选择均落在BM25-Qwen边界，强制引入静态嵌入会降低效果。留一数据集外选择始终取等权重的BM25-Qwen，在每个留出任务上均优于跨域选定的单个检索器。结果支持双检索器词法-变压器架构作为所评估荷兰任务的稳健默认方案，并表明独立基准性能不足以确立混合检索中的边际价值。
原文摘要 · Abstract (English)
Embedding benchmarks measure standalone model quality, but they do not establish whether a low-cost retriever contributes complementary ranking information once lexical and transformer-based retrieval are already combined. We present a controlled evaluation of this question across Dutch retrieval tasks from the Massive Text Embedding Benchmark for Dutch (MTEB-NL). Weighted reciprocal rank fusion (RRF) combines Best Matching 25 (BM25), Qwen/Qwen3-Embedding-0.6B (Qwen), and two multilingual static embedding models. Five datasets comprising 14,500 queries and 786,573 documents are scored exhaustively, and fusion weights are searched on a simplex in increments of 0.1. Ten-fold query-level cross-validation selects weights on nine folds and evaluates them on the held-out fold; paired bootstrap confidence intervals and sign-randomisation tests quantify the resulting differences. Fusion improves over the training-selected individual retriever by 0.061 mean reciprocal rank (MRR) on Dutch News, 0.029 on VABB, 0.004 on WebFAQ NL, and 0.025 on Wikipedia NL, while matching BM25 on Open Tender. All four positive differences remain distinguishable from zero after Holm correction. No unrestricted fold assigns positive weight to either static retriever: all 50 selections lie on the BM25-Qwen edge, and forcing a static contribution reduces effectiveness. Leave-one-dataset-out selection chooses equal BM25-Qwen weighting in every iteration and outperforms the cross-domain-selected individual retriever on every held-out task. The results support a two-retriever lexical-transformer architecture as a robust tested default across the evaluated Dutch tasks and show that standalone benchmark performance is insufficient to establish marginal value in hybrid retrieval.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。