arXiv:2605.06982cs.LG2026-05

提出并行化Tsetlin机嵌入模型,训练速度提升5倍且硬件占用小。

FastOmniTMAE: Parallel Clause Learning for Scalable and Hardware-Efficient Tsetlin Embeddings

论文配图:FastOmniTMAE: Parallel Clause Learning for Scalable and Hardware-Efficient Tsetlin Embeddings
图 1 · 摘自论文原文
  • 用两阶段并行机制替代串行训练,加速模型学习。
  • 分类任务训练快5倍,相似度指标与原模型相当。
  • 可在FPGA等资源受限设备上高效运行,适合边缘部署。

自然语言处理中的嵌入模型日益依赖深层架构如BERT,而像Word2Vec这类简单模型虽高效但可解释性差。Tsetlin机(TM)提供了一种基于逻辑的学习范式。Omni TM自动编码器(Omni TM-AE)利用单个短语层中的自动机状态分布实现静态嵌入,但其训练过程仍较慢。本文提出FastOmniTMAE,对Omni TM-AE进行重构,将串行训练依赖替换为两阶段并行流程:评估与更新。在涵盖分类、相似性和聚类的单次运行多环境基准测试中,FastOmniTMAE在分类任务上实现最高5×的训练加速,同时在Spearman和Kendall相似度度量下保持相近的嵌入质量。为解决传统GPU上TM训练效率低的问题,我们进一步在SoC-FPGA平台上实现FastOmniTMAE作为可复用加速器。多硬件基准测试显示,该模型在资源受限的FPGA上取得0.669的相似度得分,在UltraScale+ SoC上达到0.696,验证了其在小硬件开销下高效实现基于逻辑的嵌入训练能力。

原文摘要 · Abstract (English)

Embedding models in natural language processing (NLP) increasingly rely on deep architectures such as BERT, while simpler models such as Word2Vec provide efficient representations but limited interpretability. The Tsetlin Machine (TM) offers an alternative logic-based learning paradigm. Omni TM Autoencoder (Omni TM-AE) applies this paradigm to static embedding by exploiting automaton state distributions within a single clause layer, but its training process remains slow. In this work, we propose FastOmniTMAE, a reformulation of Omni TM-AE that replaces sequential training dependencies with a two-stage parallel process: evaluation and update. Using a Single-Run Multi-Environment Benchmark covering classification, similarity, and clustering, FastOmniTMAE achieves up to 5$\times$ faster training in classification while maintaining comparable embedding quality under both Spearman and Kendall similarity measures. To address the limited efficiency of TM training on conventional GPUs, we further implement FastOmniTMAE as a reusable accelerator on SoC-FPGA platforms. The Multi-Hardware Benchmark shows that FastOmniTMAE achieves similarity scores of 0.669 on a resource-constrained FPGA and 0.696 on an UltraScale+ SoC, demonstrating efficient logic-based embedding training with a small hardware footprint.

Tsetlin机嵌入模型FPGA加速并行训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。