arXiv:2412.09165cs.CLcs.AI2024-12综述被引 40

梳理大模型如何与文本嵌入融合,三类核心路径一文讲清。

When Text Embedding Meets Large Language Model: A Comprehensive Survey

  • 分三类整合路径:用大模型增强传统嵌入、让大模型直接生成嵌入、用大模型解析嵌入。
  • 揭示预训练模型时代遗留问题,指出大模型带来的新挑战。
  • 适合关注大模型与嵌入协同演进的研究者和应用开发者。

文本嵌入在深度学习时代已成为自然语言处理的基础技术,推动了众多下游任务的发展。尽管许多自然语言理解问题可通过生成范式建模并利用大语言模型(LLMs)的生成与理解能力解决,但语义匹配、聚类和信息检索等实际应用仍依赖文本嵌入的高效性与有效性。因此,将大语言模型与文本嵌入结合成为近年研究重点。本文将两者交互归纳为三大主题:(1) 大模型增强文本嵌入,用大模型改进传统嵌入方法;(2) 大模型作为文本嵌入器,利用其固有能力生成高质量嵌入;(3) 用大模型理解文本嵌入,分析与解释嵌入表示。通过按交互模式而非具体应用场景组织近期工作,提供了一个新颖且系统的视角,涵盖不同研究与应用领域的贡献。此外,本文还指出预训练语言模型时代遗留的未解决问题,并探讨大模型带来的新兴挑战。基于此分析,提出文本嵌入未来演进的潜在方向,回应当前快速发展的自然语言处理领域中的理论与实践机遇。

原文摘要 · Abstract (English)

Text embedding has become a foundational technology in natural language processing (NLP) during the deep learning era, driving advancements across a wide array of downstream tasks. While many natural language understanding challenges can now be modeled using generative paradigms and leverage the robust generative and comprehension capabilities of large language models (LLMs), numerous practical applications - such as semantic matching, clustering, and information retrieval - continue to rely on text embeddings for their efficiency and effectiveness. Therefore, integrating LLMs with text embeddings has become a major research focus in recent years. In this survey, we categorize the interplay between LLMs and text embeddings into three overarching themes: (1) LLM-augmented text embedding, enhancing traditional embedding methods with LLMs; (2) LLMs as text embedders, adapting their innate capabilities for high-quality embedding; and (3) Text embedding understanding with LLMs, leveraging LLMs to analyze and interpret embeddings. By organizing recent works based on interaction patterns rather than specific downstream applications, we offer a novel and systematic overview of contributions from various research and application domains in the era of LLMs. Furthermore, we highlight the unresolved challenges that persisted in the pre-LLM era with pre-trained language models (PLMs) and explore the emerging obstacles brought forth by LLMs. Building on this analysis, we outline prospective directions for the evolution of text embedding, addressing both theoretical and practical opportunities in the rapidly advancing landscape of NLP.

大模型文本嵌入综述NLP

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。