用几何变换解释Transformer的语义替换,让黑箱模型可读。
LAG-XAI: A Lie-Inspired Affine Geometric Framework for Interpretable Paraphrasing in Transformer Latent Spaces
- 将语义改写建模为嵌入空间中的连续仿射变换。
- 在微博数据上实现0.7713的AUC,保留80%非线性模型能力。
- 可检测大模型幻觉,95.3%错误被几何异常捕获。
现代基于Transformer的语言模型在自然语言处理任务中表现优异,但其隐含语义空间仍难以解释。本文提出LAG-XAI(Lie仿射几何解释AI),将改写视为嵌入空间中的结构化仿射变换,而非离散词替换。通过类局部李群作用的均值场近似,将改写过程分解为旋转、形变和平移等可解释几何成分。在噪声干扰的PIT-2015微博语料上,使用Sentence-BERT编码后,发现“线性透明性”现象:所提仿射算子达到AUC 0.7713,相较于随机猜测(AUC 0.5)捕捉约80%非线性基线模型的有效分类能力(AUC 0.8405),以小幅精度损失换取明确参数可解释性。模型识别出关键几何不变量:稳定矩阵重构角约27.84°,形变趋近于零,表明局部保距性。跨领域泛化通过独立TURL数据集直接验证。此外,实践证明该方法可用于大模型幻觉检测:通过“低成本几何检查”,在HaluEval数据集上自动识别95.3%的事实扭曲,仅需检测超出允许语义范围的偏差。
原文摘要 · Abstract (English)
Modern Transformer-based language models achieve strong performance in natural language processing tasks, yet their latent semantic spaces remain largely uninterpretable black boxes. This paper introduces LAG-XAI (Lie Affine Geometry for Explainable AI), a novel geometric framework that models paraphrasing not as discrete word substitutions, but as a structured affine transformation within the embedding space. By conceptualizing paraphrasing as a continuous geometric flow on a semantic manifold, we propose a computationally efficient mean-field approximation, inspired by local Lie group actions. This allows us to decompose paraphrase transitions into geometrically interpretable components: rotation, deformation, and translation. Experiments on the noisy PIT-2015 Twitter corpus, encoded with Sentence-BERT, reveal a "linear transparency" phenomenon. The proposed affine operator achieves an AUC of 0.7713. By normalizing against random chance (AUC 0.5), the model captures approximately 80% of the non-linear baseline's effective classification capacity (AUC 0.8405), offering explicit parametric interpretability in exchange for a marginal drop in absolute accuracy. The model identifies fundamental geometric invariants, including a stable matrix reconfiguration angle (~27.84°) and near-zero deformation, indicating local isometry. Cross-domain generalization is confirmed via direct cross-corpus validation on an independent TURL dataset. Furthermore, the practical utility of LAG-XAI is demonstrated in LLM hallucination detection: using a "cheap geometric check," the model automatically detected 95.3% of factual distortions on the HaluEval dataset by registering deviations beyond the permissible semantic corridor. This approach provides a mathematically grounded, resource-efficient path toward the mechanistic interpretability of Transformers.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。