通过约束句向量模长提升无监督句子嵌入效果
TNCSE: Tensor's Norm Constraints for Unsupervised Contrastive Learning of Sentence Embeddings
- 引入张量模长约束，同时优化句向量方向与长度
- 在7个语义相似度任务上达到当前最佳性能
- 适合需要高质量句向量的下游应用
无监督句子嵌入表示已成为自然语言处理中的热点研究方向。作为张量，句子嵌入具有方向和模长两个关键属性。现有方法仅关注样本表示的方向约束，忽视了模长特征。为此，我们提出一种新训练目标，通过约束正样本间的模长特征来优化无监督对比学习的训练过程。将张量模长约束与集成学习结合，构建新的句子嵌入表示框架TNCSE。在7个语义文本相似度任务上进行评估，结果表明TNCSE及其衍生模型为当前最优方法；此外，通过大量零样本评估，TNCSE表现优于其他基线模型。
原文摘要 · Abstract (English)
Unsupervised sentence embedding representation has become a hot research topic in natural language processing. As a tensor, sentence embedding has two critical properties: direction and norm. Existing works have been limited to constraining only the orientation of the samples' representations while ignoring the features of their module lengths. To address this issue, we propose a new training objective that optimizes the training of unsupervised contrastive learning by constraining the module length features between positive samples. We combine the training objective of Tensor's Norm Constraints with ensemble learning to propose a new Sentence Embedding representation framework, TNCSE. We evaluate seven semantic text similarity tasks, and the results show that TNCSE and derived models are the current state-of-the-art approach; in addition, we conduct extensive zero-shot evaluations, and the results show that TNCSE outperforms other baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。