TFA芯片用INT8量化加速Transformer推理，实现零误差翻译。
Transformer Accelerator (TFA): A Macro-Op INT8 Hardware Chip for Transformer Inference and Machine Translation
- 用8个512位宏操作描述符统一处理矩阵乘、Softmax等运算
- 在10条多语言谚语上执行7万+指令，输出与浮点参考模型完全一致
- 支持编译后量化，适合部署低功耗端侧大模型推理
我们提出Transformer加速器（TFA），一种可综合、可参数化的INT8内存到内存引擎，用于Transformer推理。单路时分复用数据通路兼顾提示处理与自回归生成。TFA通过八个512位宏操作描述符实现矩阵乘、Softmax、RMSNorm、逐元素运算及拷贝/收集操作。离线编译程序经AXI接口获取、验证并分发，支持编码器、解码器及编码器-解码器模型。RTL集成输出驻留乘加阵列与乒乓缓冲区，重叠DMA与计算；包含比特精确的倒数平方根与除法单元、键值缓存与嵌入地址映射、安全中断的零填充写入引擎。基于UVM环境与比特精确金模型进行字节级比对，在25项测试和34次约束随机运行中实现零不匹配、100%功能覆盖率和94.96%代码覆盖率。我们为t5-small编码器-解码器流水线编译了英译法、德、罗马尼亚语任务。在十个多语言谚语上，TFA执行70,320个描述符，与37.9 MB金模型输出零误差匹配。INT8输出在五句话上与浮点参考模型逐标记一致，其余产生有效替代翻译。随机哈达玛重参数化在各层恢复约11 dB每张量的INT8信噪比。验证配置相比22线程CPU实现约20倍端到端加速，更大设计预计可降低每标记能耗约1000倍。经过内存推理重编码后，逻辑面积降至2.73 mm²，设计完成SkyWater sky130工艺的设计规则清洁综合与布局布线。TFA展示了使用紧凑硬件与编译器管理量化实现预训练Transformer端到端、比特精确执行。
原文摘要 · Abstract (English)
We present the Transformer Accelerator (TFA), a synthesizable, parameterizable INT8 memory-to-memory engine for transformer inference. One time-multiplexed datapath handles prompt processing and autoregressive generation. TFA implements matrix multiplication, softmax, RMSNorm, elementwise, and copy/gather operations through eight 512-bit macro-op descriptors. Offline-compiled programs are fetched, validated, and dispatched through AXI interfaces, supporting encoder, decoder, and encoder-decoder models. The RTL combines an output-stationary multiply-accumulate array with ping-pong buffers that overlap DMA and compute, bit-exact reciprocal-square-root and divide units, key-value-cache and embedding addressing, and an abort-safe zero-padding write engine. A UVM environment byte-compares outputs against a bit-exact golden model. Across 25 tests and 34 constrained-random runs, TFA achieved zero mismatches, 100% functional coverage, and 94.96% code coverage. We compiled the t5-small encoder-decoder pipeline for English-to-French, German, and Romanian translation. On ten multilingual proverbs, TFA executed 70,320 descriptors and matched 37.9 MB of golden-model output with zero mismatches. INT8 output matched the floating-point reference token-for-token on five sentences; the rest produced valid alternative translations. Randomized-Hadamard reparameterization recovered about 11 dB of per-tensor INT8 signal-to-noise ratio across layers. The verification configuration achieved about 20x end-to-end speedup over a 22-thread CPU, while larger designs are projected to reduce energy per token by about 1000x. After RAM inference recoding, logic area fell to 2.73 mm2, and the design completed design-rule-clean synthesis and place-and-route on SkyWater sky130. TFA demonstrates end-to-end, bit-exact execution of pretrained transformers using compact hardware and compiler-managed quantization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。