利用大模型生成语义不变的句子变体，提升无训练句向量性能
GenEOL: Harnessing the Generative Power of LLMs for Training-Free Sentence Embeddings
- 用大模型生成语义一致的句子变体并聚合嵌入
- 在STS基准上平均提升2.85分，优于现有方法
- 无需训练，对提示扰动鲁棒，适合快速部署
无训练句向量方法直接利用预训练大语言模型（LLM）进行文本嵌入，避免了对比学习的高成本与复杂性。以往方法主要关注优化嵌入提示，却忽视了利用LLM的生成能力。本文提出GenEOL，通过LLM生成语义保持的句子多样化变体，并聚合这些变体的嵌入以增强整体表示。GenEOL在多个LLM上于句子语义相似度（STS）基准平均领先2.85分；在MTEB基准的聚类、重排序和配对分类任务中也取得显著提升。此外，该方法在不同LLM层间表现更稳定，对嵌入提示扰动具有强鲁棒性。
原文摘要 · Abstract (English)
Training-free embedding methods directly leverage pretrained large language models (LLMs) to embed text, bypassing the costly and complex procedure of contrastive learning. Previous training-free embedding methods have mainly focused on optimizing embedding prompts and have overlooked the benefits of utilizing the generative abilities of LLMs. We propose a novel method, GenEOL, which uses LLMs to generate diverse transformations of a sentence that preserve its meaning, and aggregates the resulting embeddings of these transformations to enhance the overall sentence embedding. GenEOL significantly outperforms the existing training-free embedding methods by an average of 2.85 points across several LLMs on the sentence semantic text similarity (STS) benchmark. GenEOL also achieves notable gains in clustering, reranking, and pair-classification tasks from the MTEB benchmark. Additionally, GenEOL stabilizes representation quality across LLM layers and remains robust to perturbations of embedding prompts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。