将S4模型压缩后部署到忆阻器存内计算硬件，实现低功耗长序列处理。
IMSSA: Deploying modern state-space models on memristive in-memory compute hardware
- 通过感知量化的训练方法，将S4D模型权重压缩至三值
- 在忆阻器交叉阵列上实现S4核的存内计算，降低内存与功耗
- 首次在存内计算硬件上部署S4模型，适合边缘设备实时推理
处理长时序序列是深度学习的关键挑战。近年来，Transformer虽为最优方案，但因需显式存储序列而内存开销巨大。结构化状态空间模型（S4）通过固定状态大小实现长序列处理，其递归线性更新机制在现代GPU上可通过卷积展开高效计算，但依赖高内存与大规模并行，仅限最新GPU支持。本文旨在将S4模型应用于边缘硬件，通过感知量化的训练方法显著压缩S4D模型的规模与计算需求，甚至实现三值权重，适用于实际任务。我们扩展了传统量化感知训练以适配模拟存内计算硬件，并成功在忆阻器交叉阵列上部署递归S4D核，实现真正的存内计算。据我们所知，这是首个在存内计算硬件上实现S4核的工作。
原文摘要 · Abstract (English)
Processing long temporal sequences is a key challenge in deep learning. In recent years, Transformers have become state-of-the-art for this task, but suffer from excessive memory requirements due to the need to explicitly store the sequences. To address this issue, structured state-space sequential (S4) models recently emerged, offering a fixed memory state while still enabling the processing of very long sequence contexts. The recurrent linear update of the state in these models makes them highly efficient on modern graphics processing units (GPU) by unrolling the recurrence into a convolution. However, this approach demands significant memory and massively parallel computation, which is only available on the latest GPUs. In this work, we aim to bring the power of S4 models to edge hardware by significantly reducing the size and computational demand of an S4D model through quantization-aware training, even achieving ternary weights for a simple real-world task. To this end, we extend conventional quantization-aware training to tailor it for analog in-memory compute hardware. We then demonstrate the deployment of recurrent S4D kernels on memrisitve crossbar arrays, enabling their computation in an in-memory compute fashion. To our knowledge, this is the first implementation of S4 kernels on in-memory compute hardware.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。