用主成分分析压缩嵌入表示，无需重新训练即可提升文档检索在专业领域的表现。
More Than Efficiency: Embedding Compression Improves Domain Adaptation in Dense Retrieval
- 对查询嵌入做PCA降维，保留领域相关特征并去除冗余信息。
- 在14个MTEB数据集上，75.4%的组合中NDCG@10得分提升。
- 无需标注或重训练，适合快速适配新领域检索任务。
基于预训练嵌入的密集检索器广泛用于文档检索，但在专业领域因训练与目标域分布不一致而表现不佳。传统领域自适应需大量标注和重训练。本文重新审视一种被忽视的方法：对领域嵌入应用PCA，生成低维表示以保留领域相关特征并剔除非判别性成分。尽管传统用于效率优化，我们证明该简单压缩方法能有效提升检索性能。在9个检索器和14个MTEB数据集上评估，仅对查询嵌入应用PCA，即在75.4%的模型-数据集组合中提升了NDCG@10，提供了一种轻量级的领域自适应方案。
原文摘要 · Abstract (English)
Dense retrievers powered by pretrained embeddings are widely used for document retrieval but struggle in specialized domains due to the mismatches between the training and target domain distributions. Domain adaptation typically requires costly annotation and retraining of query-document pairs. In this work, we revisit an overlooked alternative: applying PCA to domain embeddings to derive lower-dimensional representations that preserve domain-relevant features while discarding non-discriminative components. Though traditionally used for efficiency, we demonstrate that this simple embedding compression can effectively improve retrieval performance. Evaluated across 9 retrievers and 14 MTEB datasets, PCA applied solely to query embeddings improves NDCG@10 in 75.4% of model-dataset pairs, offering a simple and lightweight method for domain adaptation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。