提出TriADA架构，高效加速三维张量变换，提升能效与可扩展性。
TriADA: Massively Parallel Trilinear Matrix-by-Tensor Multiply-Add Algorithm and Device Architecture for the Acceleration of 3D Discrete Transformations
- 设计低秩三线性算法，实现三维离散正交变换的并行计算。
- 在30秒内完成1024×1024×1024张量变换，比传统方法快10倍以上。
- 适合高维张量密集计算场景，如AI训练和高性能科学计算。
多线性变换是高性能计算（HPC）和人工智能（AI）中的核心运算，数据以张量形式表示。然而，其计算与内存需求随维度增长而急剧上升，常导致关键任务变慢。扩大并行处理单元数量虽能提升算力，但显著增加能耗，尤其对稀疏数据（广泛存在于HPC与AI中）不利。本文提出三线性算法及同构设备架构（TriADA），通过四项创新解决该问题：（1）一种大规模并行、低秩的三线性（3D）离散正交变换（3D-DXTs）算法，属于更通用的三模矩阵-张量乘加（3D-GEMT）的特例；（2）基于外积的新GEMM核，采用解耦流式活跃内存，专为加速3D-GEMT设计；（3）与算法同构的全分布式3D网格互连处理单元网络，支持坐标无关、数据驱动的本地计算，独立于问题规模；（4）弹性稀疏外积（ESOP）方法，避免零值操作的无效计算与通信，提升能效、精度与稳定性。TriADA可在线性时间步数内完成多种三线性变换，具备超立方体复杂度下的高效计算能力。其大规模并行、可扩展且节能的架构，非常适合加速AI与HPC中最耗资源的多线性张量运算。
原文摘要 · Abstract (English)
Multilinear transformations are key in high-performance computing (HPC) and artificial intelligence (AI) workloads, where data is represented as tensors. However, their high computational and memory demands, which grow with dimensionality, often slow down critical tasks. Moreover, scaling computation by enlarging the number of parallel processing units substantially increases energy consumption, limiting widespread adoption, especially for sparse data, which is common in HPC and AI applications. This paper introduces the Trilinear Algorithm and isomorphic to algorithm Device Architecture (TriADA) to address these challenges with the following innovations: (1) a massively parallel, low-rank algorithm for computing a family of trilinear (3D) discrete orthogonal transformations (3D-DXTs), which is a special case of the more general 3-mode matrix-by-tensor multiplication (3D-GEMT); (2) a new outer-product-based GEMM kernel with decoupled streaming active memory, specially designed to accelerate 3D-GEMT operation; (3) an isomorphic to the proposed algorithm, fully distributed 3D network of mesh interconnected processing elements or cells with a coordinate-free, data-driven local processing activity, which is independent of problem size; (4) an elastic sparse outer-product (ESOP) method that avoids unnecessary computing and communication operations with zero-valued operands, thereby enhancing energy efficiency, computational accuracy, and stability. TriADA is capable of performing a variety of trilinear transformations with hypercubic arithmetic complexity in a linear number of time-steps. The massively parallel, scalable, and energy-efficient architecture of TriADA is ideal for accelerating multilinear tensor operations, which are the most demanding parts of AI and HPC workloads.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。