通过去除嵌入向量中的语言身份信号，提升多语言检索效果。
LANGSAE EDITING: Improving Multilingual Information Retrieval via Post-hoc Language Identity Removal
- 用稀疏自编码器识别并抑制嵌入向量中的语言特征
- 在多语言数据集上显著提升排名质量与跨语言覆盖
- 无需重训模型或重新编码文本，适配现有数据库
多语言密集检索常在混合语言语料中进行，但多语言嵌入同时编码了语义与语言身份信息。这种语言信号会导致同语言对相似度虚高，并掩盖其他语言的相关内容。我们提出 LANGSAE EDITING，一种基于合并嵌入训练的后处理稀疏自编码器，可在向量空间中可控地移除语言身份信号。该方法利用跨语言激活统计识别语言相关隐单元，在推理时抑制这些单元，并重建原始维度的嵌入，兼容现有向量数据库，无需重训练基础编码器或重新编码文本。多语言实验显示，该方法在排名质量与跨语言覆盖上均有持续提升，尤其在文字系统差异大的语言上表现突出。
原文摘要 · Abstract (English)
Dense retrieval in multilingual settings often searches over mixed-language collections, yet multilingual embeddings encode language identity alongside semantics. This language signal can inflate similarity for same-language pairs and crowd out relevant evidence written in other languages. We propose LANGSAE EDITING, a post-hoc sparse autoencoder trained on pooled embeddings that enables controllable removal of language-identity signal directly in vector space. The method identifies language-associated latent units using cross-language activation statistics, suppresses these units at inference time, and reconstructs embeddings in the original dimensionality, making it compatible with existing vector databases without retraining the base encoder or re-encoding raw text. Experiments across multiple languages show consistent improvements in ranking quality and cross-language coverage, with especially strong gains for script-distinct languages.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。