arXiv:2604.11615cs.ARcs.AI2026-04中稿 · DAC 2026

提出轻量统一的矩阵扩展架构,适配多种CPU并显著提升AI性能。

CUTEv2: Unified and Configurable Matrix Extension for Diverse CPU Architectures with Minimal Design Overhead

  • 解耦矩阵单元与流水线,支持低开销跨平台集成。
  • 四平台实测矩阵利用率超90%,在大模型上提速达2.31倍。
  • 异步抽象简化软件开发,适合开源生态中的硬件加速设计。

矩阵扩展已成为现代CPU应对人工智能工作负载的关键特性,但现有设计常带来高昂软硬件开销。其与CPU流水线紧密耦合,导致在多样化CPU上集成困难;细粒度同步指令也阻碍高性能内核开发。本文提出一种统一且可配置的CPU矩阵扩展架构,通过将矩阵单元与流水线解耦,在保持与现有计算和内存资源紧密协同的同时,实现低开销集成。可配置矩阵单元支持混合精度运算,适应不同计算需求与内存带宽限制。采用灵活粒度的异步矩阵乘法抽象,隐藏硬件细节,简化矩阵-向量重叠,支持统一软件栈。该架构已集成至四个开源CPU RTL平台,并在代表性AI模型上评估。在GEMM工作负载下,各平台矩阵单元利用率均超过90%。当配置为与Intel AMX相当的计算吞吐与内存带宽时,对ResNet、BERT和Llama3分别获得1.57倍、1.57倍和2.31倍加速,其中超30%增益来自矩阵-向量重叠执行。一个4 TOPS@2GHz的矩阵单元在14nm CMOS中仅占0.53 mm²。结果表明该设计具备强跨平台适应性与高效的软硬件协同优化能力,为开源社区提供实用的矩阵扩展方案。

原文摘要 · Abstract (English)

Matrix extensions have emerged as an essential feature in modern CPUs to address the surging demands of AI workloads. However, existing designs often incur substantial hardware and software design overhead. Tight coupling with the CPU pipeline complicates integration across diverse CPUs, while fine-grained synchronous instructions hinder the development of high-performance kernels. This paper proposes a unified and configurable CPU matrix extension architecture. By decoupling matrix units from the CPU pipeline, the design enables low-overhead integration while maintaining close coordination with existing compute and memory resources. The configurable matrix unit supports mixed-precision operations and adapts to diverse compute demands and memory bandwidth constraints. An asynchronous matrix multiplication abstraction with flexible granularity conceals hardware details, simplifies matrix-vector overlap, and supports a unified software stack. The architecture is integrated into four open-source CPU RTL platforms and evaluated on representative AI models. Matrix unit utilization under GEMM workloads exceeds 90% across all platforms. When configured with compute throughput and memory bandwidth comparable to Intel AMX, our design achieves speedups of 1.57x, 1.57x, and 2.31x on ResNet, BERT, and Llama3, with over 30% of the gains attributed to overlapped matrix-vector execution. A 4 TOPS@2GHz matrix unit occupies only 0.53 mm\textsuperscript{2} in 14nm CMOS. These results demonstrate strong cross-platform adaptability and effective hardware-software co-optimization, offering a practical matrix extension for the open-source community.

CPU加速矩阵扩展开源硬件异步计算

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。