提出新相似度度量DDR,能更好区分语义相近与相远的句子。
Distance-to-Distance Ratio: A Similarity Measure for Sentences Based on Rate of Change in LLM Embeddings
- 基于语言模型嵌入的变化率设计新相似度度量
- 在替换1~3个词的微小改动下仍能准确区分语义
- 适合需要精细语义判断的应用场景
文本嵌入间的相似度度量只有符合人类对文本相似性的感知才算合理。本文提出一种新的句子嵌入相似度度量——距离-距离比(DDR),受利普希茨连续性启发,通过测量前文词嵌入间相似度与后文语言模型嵌入间相似度的变化率,来衡量上下文的语义影响。我们在一系列扰动实验中评估了DDR性能,使用句子数据集生成替换1、2或3个词的变体:用同义词替换构成语义相近文本,用随机词替换构成语义相远文本。与现有主流相似度度量对比,结果表明,即使在最小且受控的编辑条件下,DDR也能持续提供更精细的区分能力。
原文摘要 · Abstract (English)
A measure of similarity between text embeddings can be considered adequate only if it adheres to the human perception of similarity between texts. In this paper, we introduce the distance-to-distance ratio (DDR), a novel measure of similarity between LLM sentence embeddings. Inspired by Lipschitz continuity, DDR measures the rate of change in similarity between the pre-context word embeddings and the similarity between post-context LLM embeddings, thus measuring the semantic influence of context. We evaluate the performance of DDR in experiments designed as a series of perturbations applied to sentences drawn from a sentence dataset. For each sentence, we generate variants by replacing one, two, or three words with either synonyms, which constitute semantically similar text, or randomly chosen words, which constitute semantically dissimilar text. We compare the performance of DDR with other prevailing similarity metrics and demonstrate that DDR consistently provides finer discrimination between semantically similar and dissimilar texts, even under minimal, controlled edits.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。