在嵌入式FPGA上自动部署极小Transformer，实现低功耗高效率时序分析。
Automating Versatile Time-Series Analysis with Tiny Transformers on Embedded FPGAs
- 结合量化训练与硬件感知搜索，自动优化模型与硬件配置。
- 4比特量化下推理功耗低至0.033mJ，延迟毫秒级，支持三类时序任务。
- 适合边缘设备开发者快速部署高效时序分析模型。
基于Transformer的模型在多种时序任务中表现优异，但其在资源受限设备上的部署仍面临内存与计算需求高的挑战。尽管已有针对微控制器（MCUs）的工作探索硬件特化优化，但这些方法通常任务特定且仅支持8位定点精度。现场可编程门阵列（FPGAs）提供更高灵活性，可精细控制数据精度与架构。然而，现有FPGA上Transformer的时序分析部署多集中于高密度平台且需手动配置。本文提出一个统一、全自动的嵌入式FPGA上极小Transformer部署框架。该框架支持一种紧凑的仅编码器结构，覆盖时序预测、分类与异常检测三类任务。通过4比特量化感知训练、基于Optuna的硬件感知超参数搜索及自动生成VHDL代码，实现无缝部署。我们在两个嵌入式FPGA平台（AMD Spartan-7与Lattice iCE40）上对六个公开数据集进行评估，结果表明：本框架生成全整数型、任务定制化加速器，在Spartan-7上实现每推理仅0.033mJ功耗与毫秒级延迟，并提供iCE40部署可行性分析。所有源码将开源至GitHub仓库（https://github.com/Edwina1030/TinyTransformer4TS）。
原文摘要 · Abstract (English)
Transformer-based models have shown strong performance across diverse time-series tasks, but their deployment on resource-constrained devices remains challenging due to high memory and computational demand. While prior work targeting Microcontroller Units (MCUs) has explored hardware-specific optimizations, such approaches are often task-specific and limited to 8-bit fixed-point precision. Field-Programmable Gate Arrays (FPGAs) offer greater flexibility, enabling fine-grained control over data precision and architecture. However, existing FPGA-based deployments of Transformers for time-series analysis typically focus on high-density platforms with manual configuration. This paper presents a unified and fully automated deployment framework for Tiny Transformers on embedded FPGAs. Our framework supports a compact encoder-only Transformer architecture across three representative time-series tasks (forecasting, classification, and anomaly detection). It combines quantization-aware training (down to 4 bits), hardware-aware hyperparameter search using Optuna, and automatic VHDL generation for seamless deployment. We evaluate our framework on six public datasets across two embedded FPGA platforms. Results show that our framework produces integer-only, task-specific Transformer accelerators achieving as low as 0.033 mJ per inference with millisecond latency on AMD Spartan-7, while also providing insights into deployment feasibility on Lattice iCE40. All source code will be released in the GitHub repository (https://github.com/Edwina1030/TinyTransformer4TS).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。