用知识图谱增强数据多样性，提升无监督句向量性能
Enhancing Unsupervised Sentence Embeddings via Knowledge-Driven Data Augmentation and Gaussian-Decayed Contrastive Learning
- 结合知识图谱与大模型生成更丰富的语义样本
- 引入高斯衰减对比学习，降低噪声负样本干扰
- 小规模数据+轻量模型即达顶尖效果，适合资源有限场景
近期利用大语言模型（LLMs）进行数据增强显著提升了无监督句向量模型性能。然而现有方法面临数据多样性不足和噪声高的双重挑战：现有策略常忽略实体、数量等细粒度知识，导致生成样本单一；同时无监督数据缺乏判别性，合成样本易引入噪声。本文提出基于管道的数据增强方法，结合知识图谱（KGs）提取实体与数量信息，指导LLM生成更多样化的样本。同时设计高斯衰减梯度辅助的对比句向量模型（GCSE），通过高斯衰减函数抑制错误硬负样本的影响，增强模型判别能力。实验表明，该方法在语义文本相似性（STS）任务中达到当前最优表现，仅需较少数据样本和较小规模的LLM，展现出高效性与鲁棒性。
原文摘要 · Abstract (English)
Recently, using large language models (LLMs) for data augmentation has led to considerable improvements in unsupervised sentence embedding models. However, existing methods encounter two primary challenges: limited data diversity and high data noise. Current approaches often neglect fine-grained knowledge, such as entities and quantities, leading to insufficient diversity. Besides, unsupervised data frequently lacks discriminative information, and the generated synthetic samples may introduce noise. In this paper, we propose a pipeline-based data augmentation method via LLMs and introduce the Gaussian-decayed gradient-assisted Contrastive Sentence Embedding (GCSE) model to enhance unsupervised sentence embeddings. To tackle the issue of low data diversity, our pipeline utilizes knowledge graphs (KGs) to extract entities and quantities, enabling LLMs to generate more diverse samples. To address high data noise, the GCSE model uses a Gaussian-decayed function to limit the impact of false hard negative samples, enhancing the model's discriminative capability. Experimental results show that our approach achieves state-of-the-art performance in semantic textual similarity (STS) tasks, using fewer data samples and smaller LLMs, demonstrating its efficiency and robustness across various models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。