交叉编码器可生成有效句向量，用于信息检索并加速推理。
Can Cross Encoders Produce Useful Sentence Embeddings?
- 利用交叉编码器早期层输出生成句向量
- 实现5.15倍推理速度提升，性能接近双编码器
- 适合需要高效检索的场景，如实时搜索系统
交叉编码器（CEs）通过句子对训练来检测相关性。传统观点认为，由于推理时需成对输入，仅能作为重排序器使用。而双编码器（DEs）虽可独立嵌入句子，但需更大数据集且精度较低。本文发现，交叉编码器早期层的输出实际上可用于信息检索。我们提出一种方法，用交叉编码器蒸馏出轻量级双编码器，实现5.15倍的推理速度提升，同时保持较高检索性能。
原文摘要 · Abstract (English)
Cross encoders (CEs) are trained with sentence pairs to detect relatedness. As CEs require sentence pairs at inference, the prevailing view is that they can only be used as re-rankers in information retrieval pipelines. Dual encoders (DEs) are instead used to embed sentences, where sentence pairs are encoded by two separate encoders with shared weights at training, and a loss function that ensures the pair's embeddings lie close in vector space if the sentences are related. DEs however, require much larger datasets to train, and are less accurate than CEs. We report a curious finding that embeddings from earlier layers of CEs can in fact be used within an information retrieval pipeline. We show how to exploit CEs to distill a lighter-weight DE, with a 5.15x speedup in inference time.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。