arXiv:2609.06155cs.IRcs.AI2026-09

让专家混合模型的检索决策变得可看懂,揭示专家路由如何影响结果排列。

ExpertLens: Visualizing Embedding Spaces for Post-Hoc Explainability in MoE Enhanced Retrievers

论文配图:ExpertLens: Visualizing Embedding Spaces for Post-Hoc Explainability in MoE Enhanced Retrievers
图 1 · 摘自论文原文
  • 用视觉化方法分析专家路由对嵌入空间的影响。
  • 实验证明专家路由使查询与相关文档更集中于几何邻域。
  • 发现通用专家主导,少数专家有语言专长,按语义相似性组织子空间。

神经网络模型在信息检索中广泛应用,性能领先但决策过程不透明。现有解释方法多关注特征重要性,难以反映嵌入空间的复杂结构。本文提出ExpertLens,一种面向专家混合(MoE)增强密集检索器的后验可解释框架,从局部特征重要性转向全局表示可解释性。该方法结合判别性嵌入空间可视化与自动生成的概念激活向量,揭示专家路由如何塑造嵌入空间与提升检索效果。在五个信息检索基准和两个MoE增强检索器上实验表明,专家路由显著改善嵌入空间结构,使查询与相关文档形成更清晰的几何邻近关系。进一步分析显示存在通用主导专家及具语言专长的少数专家,其子空间按多语义概念相似性排列。代码已公开。

原文摘要 · Abstract (English)

Neural models, including dense retrievers, have been widely adopted in Information Retrieval (IR), often delivering state-of-the-art performance. Despite their effectiveness, these models operate as black boxes, limiting the interpretability of their ranking decisions. Existing post-hoc explainability methods for neural rankers primarily focus on feature-level attributions, which can be insufficient to capture the complexity of learned embedding spaces. In this work, we propose ExpertLens, a post-hoc explainability framework for Mixture-of-Experts (MoE)-enhanced dense retrievers that shifts focus from local scalar feature importance to representation-level global interpretability. ExpertLens leverages discriminative embedding space visualizations jointly with automatically extracted Concept Activation Vectors to reveal how expert routing drives embedding space formulation and retrieval effectiveness. Experiments across five IR benchmarks and two MoE-enhanced dense retrievers show that expert routing consistently improves embedding space structure, positioning queries and their relevant documents into better-defined geometric neighborhoods. Analysis of expert subspaces further reveals general-purpose dominant experts, along with minority experts exhibiting distinct linguistic specialization, with subspaces arranged according to multi-semantic concept similarity. Our code is publicly available.

可解释性MoE嵌入空间检索器

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。