分解词向量,发现语义的细微差异
Discovering Semantic Subdimensions through Disentangled Conceptual Representations
- 将大模型词向量拆解为多个语义子向量
- 识别出可解释的语义子维度,如极性等
- 验证了这些子维度在大脑中的神经对应性
理解概念语义的核心维度对于揭示语言与大脑中意义的组织方式至关重要。现有方法通常依赖预定义的宽泛语义维度,忽略了更细粒度的概念区分。本文提出一种新框架,探究粗粒度语义维度背后的子维度。具体而言,我们引入解耦连续语义表示模型(DCSRM),将大语言模型的词向量分解为多个子向量,每个子向量编码特定语义信息。利用这些子向量,我们识别出一组可解释的语义子维度。为评估其神经合理性,我们使用体素级编码模型将这些子维度映射到大脑激活模式。研究结果提供了更具细粒度的可解释语义子维度,进一步分析表明语义维度按不同原则组织,极性是驱动其分解的关键因素。所识别子维度的神经相关性支持其认知与神经科学合理性。
原文摘要 · Abstract (English)
Understanding the core dimensions of conceptual semantics is fundamental to uncovering how meaning is organized in language and the brain. Existing approaches often rely on predefined semantic dimensions that offer only broad representations, overlooking finer conceptual distinctions. This paper proposes a novel framework to investigate the subdimensions underlying coarse-grained semantic dimensions. Specifically, we introduce a Disentangled Continuous Semantic Representation Model (DCSRM) that decomposes word embeddings from large language models into multiple sub-embeddings, each encoding specific semantic information. Using these sub-embeddings, we identify a set of interpretable semantic subdimensions. To assess their neural plausibility, we apply voxel-wise encoding models to map these subdimensions to brain activation. Our work offers more fine-grained interpretable semantic subdimensions of conceptual meaning. Further analyses reveal that semantic dimensions are structured according to distinct principles, with polarity emerging as a key factor driving their decomposition into subdimensions. The neural correlates of the identified subdimensions support their cognitive and neuroscientific plausibility.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。