arXiv:2510.00671cs.IRcs.CL2025-10被引 16

MILCO让多语言检索更高效准确,通过共享英语词空间实现跨语言精准匹配。

Milco: Learned Sparse Retrieval Across Languages via a Multilingual Connector

  • 用多语言连接器将不同语言映射到统一英语词空间,提升跨语言检索效率。
  • 在标准多语言评测中超越BGE-M3、Qwen3-Embed等主流模型,动态剪枝后仍保持高性能。
  • 支持按需剪枝,30维表示下延迟降3倍、索引缩小10倍,适合部署优化场景。

学习型稀疏检索(LSR)结合了双编码器的高效性与词汇匹配的可解释性,但现有方法难以扩展至非英文语言。本文提出MILCO,一种通过多语言连接器将查询与文档映射至共享英语词空间的LSR架构。MILCO采用两阶段训练:稀疏对齐预训练与对比学习,兼顾表示透明性与有效性,缓解语义坍塌。针对罕见实体在投影至英语时丢失的问题,提出新LexEcho头,利用特殊[ECHO]标记融合源语言视图以增强鲁棒性。MILCO在标准多语言基准上达到领先性能,优于BGE-M3、Qwen3-Embed等先进密集、稀疏及多向量基线模型。通过后处理剪枝实现动态效率:当平均仅保留30个活跃维度时,MILCO 560M优于同规模的Qwen3-Embed 0.6B(1024维),同时实现3倍更低的检索延迟与10倍更小的索引大小。

原文摘要 · Abstract (English)

Learned Sparse Retrieval (LSR) combines the efficiency of bi-encoders with the transparency of lexical matching, but existing approaches struggle to scale beyond English. We introduce MILCO, an LSR architecture that maps queries and documents from different languages into a shared English lexical space via a multilingual connector. MILCO is trained with a specialized two-stage regime that combines Sparse Alignment Pretraining with contrastive training to provide representation transparency and effectiveness while mitigating semantic collapse. Motivated by the observation that uncommon entities are often lost when projected into English, we propose a new LexEcho head, which enhances robustness by augmenting the English lexical representation with a source-language view obtained through a special [ECHO] token. MILCO achieves state-of-the-art multilingual and cross-lingual LSR performance, outperforming leading dense, sparse, and multi-vector baselines such as BGE-M3 and Qwen3-Embed on standard multilingual benchmarks, while supporting dynamic efficiency through post-hoc pruning. Notably, when using mass-based pruning to reduce document representations to only 30 active dimensions on average, MILCO 560M outperforms the similarly-sized Qwen3-Embed 0.6B with 1024 dimensions, while achieving 3$\times$ lower retrieval latency and 10$\times$ smaller index size.

多语言检索稀疏检索跨语言模型压缩

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。