MoganColBERT-TR用多向量匹配提升土耳其语检索效果，性能超越更大模型。
MoganColBERT-TR: A Late-Interaction Multi-Vector Retrieval Model for Turkish
- 采用分词级多向量表示与延迟交互打分，不压缩为单向量
- 在5个土耳其语数据集上平均nDCG@10达31.81，整体得分37.36
- 适合需要高精度检索的土耳其语应用，尤其长查询场景
我们此前提出了专为土耳其语训练的ModernBERT编码器（MoganBERT-TR）及基于其的单向量嵌入模型（MoganBERT-embed）。本文引入该系列第三款模型：MoganColBERT-TR，一种多向量检索模型，不将查询或文档压缩为单一向量，而是通过768→128投影在词元层面表示，并采用MaxSim进行延迟交互打分。该模型未从头训练：以嵌入模型的编码器为基础，经单轮蒸馏阶段适配ColBERT目标。训练数据来自两个来源——从自建预训练语料中按字符域和句边界提取的标题-段落对，以及两个土耳其语问答检索数据集；并通过交叉编码器教师（bge-reranker-v2-m3）对正样本与七个挖掘负样本的软分数进行蒸馏。实验表明，在硬负样本挖掘中，仅靠排名跳过不足，需结合组掩码与余弦上限。评估使用TurkColBERT官方流水线（PLAID索引、精确MaxSim），在五个未参与训练的土耳其语BEIR数据集上进行，均为零样本测试。模型参数量148.9M，平均得分37.36（nDCG@100: 35.53，nDCG@10: 31.81），在五种模型中排名第二：优于两倍大小的ColmmBERT-base-TR（四/五数据集领先，总体+3.05），并领先基准最大模型+12.30。与领先模型mLateOn的差距集中在查询最长的ArguAna-TR数据集。
原文摘要 · Abstract (English)
We previously reported a ModernBERT encoder trained from scratch for Turkish (MoganBERT-TR) and a single-vector embedding model built on top of it (MoganBERT-embed). This work introduces the third model in that lineage: MoganColBERT-TR, a multi-vector retrieval model that, instead of compressing a query or a document into a single vector, represents it at the token level through a 768->128 projection and scores it with MaxSim late interaction. The model is not trained from scratch: the embedding model's encoder is taken as the starting point and adapted to the ColBERT objective with a single-epoch distillation phase. Training data is produced from two sources - title-to-passage pairs carved out of our own pretraining corpus in the character domain and at sentence boundaries, and two Turkish question-based retrieval sets - and is distilled from the soft scores of a cross-encoder teacher (bge-reranker-v2-m3) over one positive and seven mined negatives. We show that in hard negative mining, rank-based skipping alone is insufficient and must be combined with a group mask and a cosine ceiling. Evaluation is carried out with the official pipeline of TurkColBERT, a benchmark built for Turkish late-interaction retrieval (PLAID index, exact MaxSim), on five Turkish BEIR datasets; none of them appears in our training pool, so all five results are clean zero-shot. With 148.9M parameters, MoganColBERT-TR reaches an overall score of 37.36 (35.53 nDCG@100, 31.81 nDCG@10) averaged over the five datasets and finishes second among the five models compared: it outperforms the twice-as-large ColmmBERT-base-TR on four of five datasets and by +3.05 overall, and the benchmark's largest model by +12.30. The gap to the leading model (mLateOn) is concentrated on ArguAna-TR, the dataset with by far the longest queries.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。