用范畴论构建可解释的词向量空间,揭示模型内在语义结构。
Transparent Semantic Spaces: A Categorical Approach to Explainable Word Embeddings
- 基于范畴论构造语义类别,实现文本语义的图形化表达。
- 证明了GloVe与Word2Vec等价于度量MDS算法,打通黑箱模型与数学框架。
- 提供计算和消除语义偏见的数学方法,适合可解释AI研究者。
本文提出一种基于范畴论的新框架,以提升人工智能系统(尤其是词向量)的可解释性。核心包括构建类别$\\(mathcal{L}_T$和$\\(mathcal{P}_T$,对文本$T$的语义进行图示表示,并将最大概率元素的选择重新定义为范畴概念。进一步构造了张量范畴$\\(mathcal{P}_T$,用于可视化从$T$中提取语义信息的多种方法,给出不依赖维度的语义空间定义,仅基于文本内部信息。同时定义了配置类别Conf与词向量类别$\\(mathcal{Emb}$,并引入散度作为$\\(mathcal{Emb}$上的装饰。建立了精确比较词向量的数学方法,证明了GloVe与Word2Vec算法等价于度量MDS算法,实现了从神经网络黑箱到透明数学框架的转化。最后提出计算嵌入前偏见的数学方法,并提供在语义空间层面缓解偏见的思路,推动可解释人工智能发展。
原文摘要 · Abstract (English)
The paper introduces a novel framework based on category theory to enhance the explainability of artificial intelligence systems, particularly focusing on word embeddings. Key topics include the construction of categories $\mathcal{L}_T$ and $\mathcal{P}_T$, providing schematic representations of the semantics of a text $ T $, and reframing the selection of the element with maximum probability as a categorical notion. Additionally, the monoidal category $\mathcal{P}_T$ is constructed to visualize various methods of extracting semantic information from $T$, offering a dimension-agnostic definition of semantic spaces reliant solely on information within the text. Furthermore, the paper defines the categories of configurations Conf and word embeddings $\mathcal{Emb}$, accompanied by the concept of divergence as a decoration on $\mathcal{Emb}$. It establishes a mathematically precise method for comparing word embeddings, demonstrating the equivalence between the GloVe and Word2Vec algorithms and the metric MDS algorithm, transitioning from neural network algorithms (black box) to a transparent framework. Finally, the paper presents a mathematical approach to computing biases before embedding and offers insights on mitigating biases at the semantic space level, advancing the field of explainable artificial intelligence.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。