arXiv:2510.09790cs.CL2025-10被引 1

发现语言变换在嵌入空间中可表示为统一旋转,跨模型跨语言通用。

Mapping Semantic & Syntactic Relationships with Geometric Rotation

  • 将语义语法变换建模为嵌入空间中的统一旋转操作。
  • 在五类语言、三模型、七语种上验证其对否定、条件等特征的精准映射。
  • 首次实证支持句级线性表征假设,适合自然语言理解研究者。

理解语言与嵌入模型如何编码语义关系是模型可解释性的核心。尽管早期词嵌入具有直观的向量运算特性(如'king' - 'man' + 'woman' = 'queen'),现代高维文本表示缺乏明确的可解释几何性质。本文提出旋转不变位移估计(RISE),一种基于几何的方法,将语义-句法变换表示为嵌入空间中一致的旋转操作,利用现代语言表示的流形结构。RISE可在不同语言和模型间通用且不降低性能,表明跨语言存在相似的几何结构。我们在三种嵌入模型、三个数据集、七种形态多样的语言(覆盖五大语言族)上对比评估RISE,结果表明其能一致地映射话语级语义-句法变换(如否定、条件性)并保留显著语法特征。该工作首次实证证明话语级语义-句法变换在多语言嵌入空间中对应一致的几何操作,为句级线性表征假设提供了经验证据。

原文摘要 · Abstract (English)

Understanding how language and embedding models encode semantic relationships is fundamental to model interpretability. While early word embeddings exhibited intuitive vector arithmetic (''king'' - ''man'' + ''woman'' = ''queen''), modern high-dimensional text representations lack straightforward interpretable geometric properties. We introduce Rotor-Invariant Shift Estimation (RISE), a geometric approach that represents semantic-syntactic transformations as consistent rotational operations in embedding space, leveraging the manifold structure of modern language representations. RISE operations have the ability to operate across both languages and models without reducing performance, suggesting the existence of analogous cross-lingual geometric structure. We compare and evaluate RISE using two baseline methods, three embedding models, three datasets, and seven morphologically diverse languages in five major language groups. Our results demonstrate that RISE consistently maps discourse-level semantic-syntactic transformations with distinct grammatical features (e.g., negation and conditionality) across languages and models. This work provides the first demonstration that discourse-level semantic-syntactic transformations correspond to consistent geometric operations in multilingual embedding spaces, empirically supporting the linear representation hypothesis at the sentence level.

语义几何嵌入空间跨语言可解释性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。