用扩散张量成像分析语言模型中词语的语义流动
Visualising Information Flow in Word Embeddings with Diffusion Tensor Imaging
- 将扩散张量成像技术应用于词向量，捕捉语义在句子中的流动
- 揭示了模型各层间语义表示的变化规律
- 适合关注模型可解释性与结构优化的研究者
理解大语言模型（LLMs）如何表征自然语言是自然语言处理研究的核心挑战。现有方法通常提取词向量，通过点图可视化嵌入空间并比较特定词语的位置，但仅关注孤立词语，忽略其使用语境。本文提出一种新工具，通过扩散张量成像（DTI）分析自然语言表达中的信息流动。结果表明，DTI能揭示嵌入空间中词元间语义表示的变化。追踪模型各层中的变化，有助于比较不同模型结构，并可能发现可剪枝的低效层。该方法为理解大语言模型如何表征真实自然语言表达提供了新视角，超越了对孤立词向量的比较，提升了NLP模型的可解释性。
原文摘要 · Abstract (English)
Understanding how large language models (LLMs) represent natural language is a central challenge in natural language processing (NLP) research. Many existing methods extract word embeddings from an LLM, visualise the embedding space via point-plots, and compare the relative positions of certain words. However, this approach only considers single words and not whole natural language expressions, thus disregards the context in which a word is used. Here we present a novel tool for analysing and visualising information flow in natural language expressions by applying diffusion tensor imaging (DTI) to word embeddings. We find that DTI reveals how embedding space representations change between tokens. Tracking these changes within the layers of an LLM allows for comparing different model structures and could potentially reveal opportunities for pruning an LLM's under-utilised layers. Our results show that our visualisation method permits novel insights into how LLMs represent actual natural language expressions, extending the comparison of isolated word embeddings and improving the interpretability of NLP models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。