优化浅层神经网络的前向与反向传播，提升GPU运行效率
GPU Parallelization Strategies for Forward and Backward Propagation in Shallow Neural Networks: A CUDA-Based Comparative Study
- 采用分块共享内存+列偏移避免冲突，提升内存访问效率
- 预转置权重矩阵实现合并内存访问，减少数据搬运
- 融合矩阵乘法与激活函数计算，避免中间存储开销
本文对比研究了应用于浅层神经网络前向与反向传播的CUDA优化策略。评估了三种叠加优化：（1）使用分块共享内存并以+1列填充消除银行冲突；（2）预转置权重矩阵以实现合并全局内存访问；（3）融合MatMul+ReLU内核，避免中间全局内存往返。在NVIDIA Tesla T4（CUDA 13.0）上对三个数据集规模进行实验，全优化版本在大数据集（25,600样本）上相比基线CUDA版本提速1.41倍，执行时间从21.0秒降至14.8秒。结果与串行CPU和OpenMP并行实现对比，验证了内存访问优化在GPU加速深度学习原语中的有效性。
原文摘要 · Abstract (English)
We present a comparative study of CUDA optimization strategies applied to forward and backward propagation in a shallow neural network. Three stacked optimizations are evaluated: (1) tiled shared memory with bank-conflict elimination via +1-column padding, (2) pre-transposed weight matrices for coalesced global memory access, and (3) a fused MatMul+ReLU kernel that eliminates intermediate global-memory round-trips. Experiments on an NVIDIA Tesla T4 (CUDA 13.0) across three dataset sizes show that the fully optimized implementation achieves a 1.41x speedup over the baseline CUDA version on the large dataset (25,600 samples), reducing execution time from 21.0s to 14.8s. Results are compared against a sequential CPU baseline and an OpenMP parallel implementation, demonstrating the effectiveness of memory-access optimization in GPU-accelerated deep learning primitives.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。