用稀疏自编码器让句子嵌入可解释，实现精准检索控制
Aligning Sentence Embeddings to Human Concepts via Sparse Autoencoders
- 用Top-k稀疏自编码器分解句子嵌入，提取人类可理解的概念
- 解耦后的特征与语义、句法、语用类别高度对齐
- 通过激活控制可重排结果，无需重训练模型
密集句子嵌入是现代检索增强生成（RAG）系统的核心，但因特征叠加而缺乏可解释性，难以对齐人类意图。本文提出使用Top-k稀疏自编码器（SAEs）将句子变压器（如E5）的密集表示分解为人类可理解的概念。我们证明这些解耦特征与特定语义、句法和语用类别高度对齐。此外，引入激活控制机制，通过固定特定潜在特征，可在不重新训练主干模型的前提下，精确干预检索过程并重排搜索结果以更好满足用户约束。结果表明，基于SAE的分解为透明且可操控的神经信息检索提供了可行路径。
原文摘要 · Abstract (English)
Dense sentence embeddings are fundamental to modern Retrieval-Augmented Generation (RAG) systems but suffer from a lack of interpretability due to feature superposition. This opacity hinders the alignment of retrieval processes with human intent, as the entangled representations are difficult to analyze or control. In this work, we propose a method to disentangle the dense representations of sentence transformers (e.g., E5) into human-interpretable concepts using Top-k Sparse Autoencoders (SAEs). We demonstrate that these disentangled features align with specific semantic, syntactic, and pragmatic categories. Furthermore, we introduce an activation steering mechanism that allows for precise intervention in the retrieval process. By clamping specific latent features, we show that it is possible to re-rank search results to better align with user constraints without retraining the backbone model. Our findings suggest that SAE-based decomposition offers a viable path toward transparent and steerable neural information retrieval.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。