找出文本间语义不同的关键词段，让相似度评分更可解释。
Explainable Semantic Textual Similarity via Dissimilar Span Detection
- 提出检测文本差异片段的新任务，定位影响相似度的具体词元。
- 构建半自动标注的跨度相似性数据集SSD，支持该任务研究。
- 实验证明该方法能提升改写检测任务性能，适合可解释性需求场景。
语义文本相似度（STS）是众多自然语言处理应用的核心组件。然而，现有方法通常将语义差异简化为单一分数，限制了可解释性。为此，我们提出新任务——不相似跨度检测（DSD），旨在识别两段文本间的语义差异部分。这有助于用户理解哪些词语或标记降低了相似度得分，也可用于提升依赖STS的下游任务表现。此外，我们通过结合大语言模型（LLMs）与人工验证的半自动化流程，构建了适用于该任务的新数据集——跨度相似性数据集（SSD）。我们提出了多种基线方法进行评估，包括基于LIME、SHAP、LLMs的无监督方法及自研方法，还包含一种监督方法。尽管LLMs和监督模型表现最佳，整体效果仍较低，凸显任务复杂性。最后，我们设计实验表明，利用DSD可显著提升改写检测任务的性能。
原文摘要 · Abstract (English)
Semantic Textual Similarity (STS) is a crucial component of many Natural Language Processing (NLP) applications. However, existing approaches typically reduce semantic nuances to a single score, limiting interpretability. To address this, we introduce the task of Dissimilar Span Detection (DSD), which aims to identify semantically differing spans between pairs of texts. This can help users understand which particular words or tokens negatively affect the similarity score, or be used to improve performance in STS-dependent downstream tasks. Furthermore, we release a new dataset suitable for the task, the Span Similarity Dataset (SSD), developed through a semi-automated pipeline combining large language models (LLMs) with human verification. We propose and evaluate different baseline methods for DSD, both unsupervised, based on LIME, SHAP, LLMs, and our own method, as well as an additional supervised approach. While LLMs and supervised models achieve the highest performance, overall results remain low, highlighting the complexity of the task. Finally, we set up an additional experiment that shows how DSD can lead to increased performance in the specific task of paraphrase detection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。