arXiv:2509.24425cs.LGcs.AR2025-09

将注意力机制融入超维计算,实现高效低延迟多变量时序分类

BiHDTrans: binary hyperdimensional transformer for efficient multivariate time series classification

  • 把自注意力机制嵌入二值超维空间,融合模型表达力与计算效率
  • 在多个数据集上比顶尖二值化Transformer高6.67%准确率,延迟降低39.4倍
  • 超维空间二值化信息失真小,64%降维下仍保持竞争力,适合边缘设备部署

物联网设备激增带来海量多变量时间序列(MTS)数据,亟需在资源受限的边缘环境中实现高效精准处理。超维(HD)计算具有天然高效和并行性优势,但难以捕捉复杂时序模式;而Transformer虽擅长序列建模,却存在高计算与内存开销。本文提出BiHDTrans——一种神经符号式二值超维Transformer,首次将自注意力机制融入超维计算范式,融合了超维表示的高效性与Transformer的时序建模能力。实验表明,BiHDTrans相比最优的超维计算模型至少提升14.47%准确率,平均比顶尖二值Transformer高出6.67%。基于FPGA的流水线实现利用高维表示的独立同分布特性,推理延迟较最优二值Transformer降低39.4倍。理论分析表明,在全息高维空间中二值化带来的信息失真远小于直接对神经网络二值化,解释了其优异性能。维度实验进一步验证:即使维度降低64%,模型仍保持竞争力,相比最先进二值化Transformer,模型规模缩小4.4倍,延迟再降49.8%。这些成果弥合了Transformer表达力与超维计算效率之间的鸿沟,实现了可扩展、低延迟、高精度的MTS分类。

原文摘要 · Abstract (English)

The proliferation of Internet-of-Things (IoT) devices has led to an unprecedented volume of multivariate time series (MTS) data, requiring efficient and accurate processing for timely decision-making in resource-constrained edge environments. Hyperdimensional (HD) computing, with its inherent efficiency and parallelizability, has shown promise in classification tasks but struggles to capture complex temporal patterns, while Transformers excel at sequence modeling but incur high computational and memory overhead. We introduce BiHDTrans, an efficient neurosymbolic binary hyperdimensional Transformer that integrates self-attention into the HD computing paradigm, unifying the representational efficiency of HD computing with the temporal modeling power of Transformers. Empirically, BiHDTrans outperforms state-of-the-art (SOTA) HD computing models by at least 14.47% and achieves 6.67% higher accuracy on average than SOTA binary Transformers. With hardware acceleration on FPGA, our pipelined implementation leverages the independent and identically distributed properties of high-dimensional representations, delivering 39.4 times lower inference latency than SOTA binary Transformers. Theoretical analysis shows that binarizing in holographic high-dimensional space incurs significantly less information distortion than directly binarizing neural networks, explaining BiHDTrans's superior accuracy. Furthermore, dimensionality experiments confirm that BiHDTrans remains competitive even with a 64% reduction in hyperspace dimensionality, surpassing SOTA binary Transformers by 1-2% in accuracy with 4.4 times less model size, as well as further reducing the latency by 49.8% compare to the full-dimensional baseline. Together, these contributions bridge the gap between the expressiveness of Transformers and the efficiency of HD computing, enabling accurate, scalable, and low-latency MTS classification.

时序分类超维计算边缘推理二值化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。