arXiv:2605.00536cs.DCcs.AR2026-05

Tempus让边缘芯片用固定核心高效跑大矩阵乘法,省电又不卡顿。

Tempus: A Temporally Scalable Resource-Invariant GEMM Streaming Framework for Versal AI Edge

  • 用固定16核+流水迭代,靠时间换空间实现可扩展性
  • 每秒处理6070亿次操作,功耗仅10.677瓦,能效比高211倍
  • 适合资源紧张的边缘设备,尤其对低功耗大模型推理友好

大语言模型的质量随计算规模提升,但边缘部署受限于算力、内存和功耗。由于通用矩阵乘法(GEMM)占推理时间高达90%,其加速至关重要。AMD Versal自适应SoC中的自适应智能引擎适配此任务,但现有最先进框架通过空间扩展(分配数百个核心)追求性能,导致物理实现失败、带宽饱和与资源过度消耗,在资源受限的边缘芯片上不可行。本文提出Tempus,一种面向AMD Versal AI Edge SoC的资源无关型时序可扩展GEMM流式框架。它不随矩阵规模增加硬件资源,而是采用固定的16个AIE-ML核心,通过迭代图执行及可编程逻辑中的算法数据分块与复制实现可扩展性。高速级联流确保初始间隔(II)为1时的低延迟部分和归约,无死锁的数据流协议最大化传输与计算重叠及PLIO复用。在GEMM工作负载上,Tempus实现607 GOPS,总片上功耗10.677 W。通过平台感知效用(PAU)度量系统级效率,证明其相比领先的空间方案(ARIES)突出因子提高211.2倍。同时,该框架保持0%的URAM/DSP利用率,实现22.0倍核心节俭、7.1倍功耗节俭和6.3倍输入输出需求减少,为边缘大模型推理建立了可持续、可扩展的基础。

原文摘要 · Abstract (English)

Scaling laws for Large Language Models (LLMs) establish that model quality improves with computational scale, yet edge deployment imposes strict constraints on compute, memory, and power. Since General Matrix Multiplication (GEMM) accounts for up to 90% of inference time, efficient GEMM acceleration is critical for edge AI. The Adaptive Intelligent Engines available in the AMD Versal adaptive SoCs are well suited for this task, but existing state-of-the-art (SOTA) frameworks maximize performance through spatial scaling, distributing workloads across hundreds of cores -- an approach that fails on resource-limited edge SoCs due to physical implementation failures, bandwidth saturation, and excessive resource consumption. We propose Tempus, a Resource-Invariant Temporal GEMM framework for the AMD Versal AI Edge SoC. Rather than expanding hardware resources with matrix size, Tempus employs a fixed compute block of 16 AIE-ML cores, achieving scalability through iterative graph execution and algorithmic data tiling and replication in the Programmable Logic. High-speed cascade streaming ensures low-latency partial sum reduction at Initiation Interval (II) of 1, while a deadlock-free DATAFLOW protocol maximizes transfer-compute overlap and PLIO reuse. Evaluated on GEMM workloads, Tempus achieves 607 GOPS at 10.677 W total on-chip power. By characterizing system-level efficiency through the Platform-Aware Utility (PAU) metric, we prove that Tempus achieves a 211.2x higher prominence factor than the leading spatial SOTA (ARIES). Furthermore, the framework maintains a 0.00% utilization of URAM/DSP, yielding 22.0x core frugality, 7.1x power frugality, and a 6.3x reduction in I/O demand, establishing a sustainable, scalable foundation for edge LLM inference.

边缘AIGEMM加速能效优化自适应芯片

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。