arXiv:2505.15507cs.LGcs.AI2025-05被引 4

提出多维组合嵌入的新代数结构,统一序列与图像建模范式。

Directional Non-Commutative Monoidal Structures for Compositional Embeddings in Machine Learning

  • 为每个维度设计非交换的组合算子,保持各维结合律
  • 各维算子彼此交换,确保跨维组合全局一致
  • 可统一变换器、状态空间模型等经典架构,适合未来模型设计

我们提出一种基于方向性非交换幺半群算子的多维组合嵌入新代数结构。该框架为每个轴 i 定义独立的组合算子 circ_i,保证各维结合律,但不强制全局交换性;同时,所有轴算子相互交换,满足全局交换律,确保跨维组合一致性。这是首个将经典序列建模范式(如结构化状态空间模型(SSMs)和变换器自注意力)统一到多维框架的方法。一维特例可恢复仿射变换、普通自注意力及 SSM 式递归;高维推广自然支持嵌入空间中的递归与结构感知操作。该结构可赋能结构化位置编码、方向性图像嵌入及网格/序列的符号建模,或指导未来深度学习架构设计。我们形式化证明了其代数性质并讨论高效实现方式。由于聚焦理论分析,本文未包含实验,实证验证将留待后续工作。

原文摘要 · Abstract (English)

We introduce a new algebraic structure for multi-dimensional compositional embeddings, built on directional non-commutative monoidal operators. The core contribution of this work is this novel framework, which exhibits appealing theoretical properties (associativity along each dimension and an interchange law ensuring global consistency) while remaining compatible with modern machine learning architectures. Our construction defines a distinct composition operator circ_i for each axis i, ensuring associative combination along each axis without imposing global commutativity. Importantly, all axis-specific operators commute with one another, enforcing a global interchange law that enables consistent crossaxis compositions. This is, to our knowledge, the first approach that provides a common foundation that generalizes classical sequence-modeling paradigms (e.g., structured state-space models (SSMs) and transformer self-attention) to a unified multi-dimensional framework. For example, specific one-dimensional instances of our framework can recover the familiar affine transformation algebra, vanilla self-attention, and the SSM-style recurrence. The higher-dimensional generalizations naturally support recursive, structure-aware operations in embedding spaces. We outline several potential applications unlocked by this structure-including structured positional encodings in Transformers, directional image embeddings, and symbolic modeling of sequences or grids-indicating that it could inform future deep learning model designs. We formally establish the algebraic properties of our framework and discuss efficient implementations. Finally, as our focus is theoretical, we include no experiments here and defer empirical validation to future work, which we plan to undertake.

嵌入空间代数结构多维建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。