针对图神经网络训练内存瓶颈，设计了基于超立方体的高效消息传递架构。
Efficient Message Passing Architecture for GCN Training on HBM-based FPGAs with Orthogonal Topology On-Chip Networks
- 采用4维超立方体拓扑实现并行多播路由，提升通信效率
- 重构反向传播算法，减少矩阵转置开销，内存占用降低30%以上
- 适配高带宽内存FPGA，适用于大规模图学习任务
图卷积网络（GCNs）是图结构数据表示学习的前沿模型。然而，其高效训练受限于内存容量与带宽瓶颈，加之不规则数据流导致通信阻塞。为此，本文提出一种基于NUMA内存访问特性的消息传递架构，在加速器内采用4维超立方体网络实现并行多播路由，以提升图上消息传递效率。同时，针对GCN特性重新设计反向传播算法，有效缓解训练阶段的内存压力，并降低大规模矩阵转置带来的计算开销。相较于当前最优的HP-GNN架构，本方案在性能上提升1.03倍至1.81倍。
原文摘要 · Abstract (English)
Graph Convolutional Networks (GCNs) are state-of-the-art deep learning models for representation learning on graphs. However, the efficient training of GCNs is hampered by constraints in memory capacity and bandwidth, compounded by the irregular data flow that results in communication bottlenecks. To address these challenges, we propose a message-passing architecture that leverages NUMA-based memory access properties and employs a parallel multicast routing algorithm based on a 4-D hypercube network within the accelerator for efficient message passing in graphs. Additionally, we have re-engineered the backpropagation algorithm specific to GCNs within our proposed accelerator. This redesign strategically mitigates the memory demands prevalent during the training phase and diminishes the computational overhead associated with the transposition of extensive matrices. Compared to the state-of-the-art HP-GNN architecture we achieved a performance improvement of $1.03\times \sim 1.81\times$.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。