提出无需固定维度的Transformer，让输入输出任意维。
On Dimension-Free Transformer: An Application of STP to AI
- 用投影构造超向量线性变换，替代传统Transformer中固定维度操作。
- 验证所有线性变换可替换为投影式超向量变换，实现任意维度输入输出。
- 适合处理多维信号，特别在信息均衡分布场景更高效。
首先描述了Transformer各部分的矩阵表达式。基于矩阵的半张量积（STP），重新审视超向量，并通过投影构造超向量上的线性变换，获得其性质与计算公式。利用基于投影的超向量变换（PBTH），通过验证Transformer中每一项线性变换并替换为合适的PBTH，提出无维度Transformer（DFT）框架，使输入和输出可为任意维度。利用所有元素的均衡信息，DFT在处理信号时应更具效率。
原文摘要 · Abstract (English)
The matrix expressions for every parts of a transformer are firstly described. Based on semi-tensor product (STP) of matrices the hypervectors are reconsidered and the linear transformation over hypervectors is constructed by using projection. Its properties and calculating formulas are obtained. Using projection-based transformation of hypervector (PBTH), the framework of dimension-free transformer (DFT) is proposed by verifying each linear transformation in a transformer and replacing it by a proper PBTH, which allows the inputs and outputs being of arbitrary dimensions. Using balanced information about all entries, DFT must be more efficient in dealing with signals.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。