用大模型生成有语义区别的句子对，提升句向量模型性能。
Refining Sentence Embedding Model through Ranking Sentences Generation with Large Language Models
- 控制大模型在隐空间的生成方向，确保句子语义差异有意义。
- 在多个基准上达到新最好结果，仅需少量排序句子生成成本。
- 适合想提升句向量质量又不想依赖人工标注的研究者。
句向量在众多自然语言处理任务中至关重要，对比学习方法在如NLI等标注数据集上表现优异。然而，依赖人工标注限制了可扩展性。近期研究利用大语言模型（LLM）生成句子对，降低标注依赖。但现有方法忽略了对细粒度语义区分至关重要的排序信息。为此，我们提出一种控制LLM生成方向的方法，使其在隐空间中生成具有意义语义差别的句子。相比无约束生成，该方法能有效保证语义分化。随后，我们通过融合排序与语义信息，对现有句向量模型进行优化。在多个基准上的实验表明，该方法以较小的排序句子生成开销，实现了新的最先进性能。
原文摘要 · Abstract (English)
Sentence embedding is essential for many NLP tasks, with contrastive learning methods achieving strong performance using annotated datasets like NLI. Yet, the reliance on manual labels limits scalability. Recent studies leverage large language models (LLMs) to generate sentence pairs, reducing annotation dependency. However, they overlook ranking information crucial for fine-grained semantic distinctions. To tackle this challenge, we propose a method for controlling the generation direction of LLMs in the latent space. Unlike unconstrained generation, the controlled approach ensures meaningful semantic divergence. Then, we refine exist sentence embedding model by integrating ranking information and semantic information. Experiments on multiple benchmarks demonstrate that our method achieves new SOTA performance with a modest cost in ranking sentence synthesis.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。