梳理词向量到多模态嵌入的演进路径，助你快速掌握大模型核心基础。
From Word Vectors to Multimodal Embeddings: Techniques, Applications, and Future Directions For Large Language Models
- 从词向量到上下文嵌入，系统梳理静态与动态表示方法的发展脉络。
- 涵盖BERT、GPT等主流模型在跨语言与个性化任务中的应用进展。
- 聚焦可解释性、偏见消减与多模态融合，适合研究者与工程实践者参考。
词向量与语言模型通过将语言元素映射到连续向量空间，彻底改变了自然语言处理。本文回顾分布假设与上下文相似性等基础概念，梳理从独热编码等稀疏表示到Word2Vec、GloVe、fastText等稠密嵌入的演变过程。重点分析静态与上下文嵌入技术，强调ELMo、BERT、GPT等模型在跨语言与个性化应用中的进步。讨论句子与文档级嵌入的聚合方法及生成式主题建模，并拓展至视觉、机器人学与认知科学等多模态领域。深入剖析模型压缩、可解释性、数值编码与偏见缓解等前沿议题，涵盖技术挑战与伦理影响。最后提出未来方向：需发展可扩展训练技术、提升可解释性，并增强非文本模态的稳健表征能力。本综述为研究人员与从业者提供嵌入式语言模型的深度资源。
原文摘要 · Abstract (English)
Word embeddings and language models have transformed natural language processing (NLP) by facilitating the representation of linguistic elements in continuous vector spaces. This review visits foundational concepts such as the distributional hypothesis and contextual similarity, tracing the evolution from sparse representations like one-hot encoding to dense embeddings including Word2Vec, GloVe, and fastText. We examine both static and contextualized embeddings, underscoring advancements in models such as ELMo, BERT, and GPT and their adaptations for cross-lingual and personalized applications. The discussion extends to sentence and document embeddings, covering aggregation methods and generative topic models, along with the application of embeddings in multimodal domains, including vision, robotics, and cognitive science. Advanced topics such as model compression, interpretability, numerical encoding, and bias mitigation are analyzed, addressing both technical challenges and ethical implications. Additionally, we identify future research directions, emphasizing the need for scalable training techniques, enhanced interpretability, and robust grounding in non-textual modalities. By synthesizing current methodologies and emerging trends, this survey offers researchers and practitioners an in-depth resource to push the boundaries of embedding-based language models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。