对比大模型与经典模型的词向量，发现大模型在语义聚类上更优但非上下文任务仍不如传统方法
Revisiting Word Embeddings in the LLM Era
- 系统比较大模型与Word2Vec、SBERT等经典模型的词向量表现
- 大模型在无上下文语义聚类和类比任务中表现更好，但上下文任务落后于SimCSE
- 揭示传统模型在细粒度语义任务中的持续价值，适合对精度要求高的场景
大型语言模型（LLMs）在自然语言处理任务中展现出显著进展，近期流行的做法是从中提取词、句或文档嵌入，并用于各类推理任务，效果令人满意。然而，这种性能提升究竟是源于模型规模，还是其生成的嵌入本身与传统的词嵌入模型如Word2Vec、GloVe、Sentence-BERT（SBERT）或Universal Sentence Encoder（USE）有本质差异，尚不明确。本文通过系统比较经典去上下文化与上下文化词嵌入，以及同源的LLM诱导嵌入，发现大模型在无上下文设置下能更紧密地聚类语义相关词汇，并在类比任务中表现更优；但在上下文设置下，如SimCSE等经典模型在句子级相似性评估任务中仍优于大模型，凸显其在细粒度语义理解中的持续重要性。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have recently shown remarkable advancement in various NLP tasks. As such, a popular trend has emerged lately where NLP researchers extract word/sentence/document embeddings from these large decoder-only models and use them for various inference tasks with promising results. However, it is still unclear whether the performance improvement of LLM-induced embeddings is merely because of scale or whether underlying embeddings they produce significantly differ from classical encoding models like Word2Vec, GloVe, Sentence-BERT (SBERT) or Universal Sentence Encoder (USE). This is the central question we investigate in the paper by systematically comparing classical decontextualized and contextualized word embeddings with the same for LLM-induced embeddings. Our results show that LLMs cluster semantically related words more tightly and perform better on analogy tasks in decontextualized settings. However, in contextualized settings, classical models like SimCSE often outperform LLMs in sentence-level similarity assessment tasks, highlighting their continued relevance for fine-grained semantics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。