arXiv:2602.00003cs.IRcs.AI2026-02中稿 · the Workshop on TI…

用多个专家模型协作提升跨境电商搜索相关性,效果更好且更快。

Orchestrating Heterogeneous Experts: A Scalable MoE Framework with Anisotropy-Preserving Fusion

  • 将不同语言专长的开源大模型动态组合,按查询分配专家处理。
  • 在六个东南亚市场数据上,AUC提升0.72个百分点,相同参数量下性能更优。
  • 适合需要多语言支持、追求高吞吐的电商搜索系统开发者。

在跨境电子商务中,搜索相关性建模面临极端语言多样性和细粒度语义差异的双重挑战。现有方法通常依赖扩大单一大型语言模型(LLM)规模,但实证分析表明,单模型在不同地区能力分布不均,例如在英语表现优异,而在特定东南亚语言上表现欠佳。本文提出从扩展单一模型转向协同异构专家的新范式。我们构建了一个可扩展的粗粒度混合专家(MoE)框架,利用不同开源大模型(如Qwen、Gemma)的内在互补性,无需昂贵预训练。与标准基于令牌的MoE不同,本框架动态将完整查询路由至专业专家,并关键性地采用信息保留拼接融合策略。理论上认为,在多模型潜在空间中保持各专家嵌入流形的独立性——而非通过加权平均压缩——对于捕捉复杂相关性信号至关重要。在覆盖六个东南亚市场的数据集上,该MoE框架在相同活跃参数量下比密集基线提升0.72个百分点的AUC;同时优化流水线实现13.72查询每秒(QPS),吞吐量提升9%。

原文摘要 · Abstract (English)

In cross-border e-commerce, search relevance modeling faces the dual challenge of extreme linguistic diversity and fine-grained semantic nuances. Existing approaches typically rely on scaling up a single monolithic Large Language Model (LLM). However, our empirical analysis reveals that single models suffer from uneven capability distributions across regions. For example, excelling in English while underperforming in specific Southeast Asian languages. In this work, we shift the paradigm from scaling a single model to orchestrating heterogeneous experts. We propose a scalable Coarse-grained Mixture-of-Experts (MoE) framework that leverages the inherent complementarity of distinct open-source LLMs (e.g., Qwen, Gemma) without expensive pre-training. Unlike standard token-level MoE, our framework dynamically routes entire queries to specialized experts and, crucially, employs an Information-Preserving Concatenation Fusion strategy. We theoretically posit that preserving the distinct embedding manifolds of heterogeneous experts-rather than compressing them via weighted averaging-is essential for capturing complex relevance signals in a multi-model latent space. On datasets spanning six Southeast Asian markets, our MoE improves AUC by 0.72 percentage points over a dense baseline with the same active parameters. Meanwhile, the optimized pipeline achieves 13.72 queries per second (QPS), a 9% throughput improvement.

混合专家多语言搜索排序

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。