改变GEMM输入数据可降低40%显卡功耗，为节能提供新思路。
Input-Dependent Power Usage in GPUs
- 通过调整输入数据分布、稀疏性等，不改矩阵尺寸即可调节能耗
- 实测显示不同输入导致显卡功耗相差近40%
- 适合关注数据中心能效与编译器优化的研究者
GPU因人工智能兴起成为数据中心主要能耗来源。主流工作负载以大规模通用矩阵乘法（GEMM）为主，其硬件利用率已高度优化。本文发现，在保持矩阵形状不变的前提下，仅修改输入数据（如值分布、位相似性、位置、稀疏性），即可显著改变GEMM内核的功耗。在多种数据类型上测试四种输入变化后，观测到功耗波动接近40%。我们推测此现象源于芯片内部比特翻转次数的变化。据此提出通过编译器与调度器优化，利用输入依赖的功耗特性来管理能源消耗。
原文摘要 · Abstract (English)
GPUs are known to be power-hungry, and due to the boom in artificial intelligence, they are currently the major contributors to the high power demands of upcoming datacenters. Most GPU usage in these popular workloads consist of large general matrix-matrix multiplications (GEMMs), which have therefore been optimized to achieve high utilization of hardware resources. In this work, we show that modifying the input data to GEMMs, while maintaining the matrix shapes and sizes can notably change the power consumption of these kernels. We experiment with four kinds of input variations: value distribution, bit similarity, placement, and sparsity, across different data types. Our findings indicate that these variations can change the GPU power usage during GEMM by almost 40%. We hypothesize that input-dependent power usage variations occur due to changes in the number of bit flips in the GPUs. We propose leveraging this property through compiler and scheduler optimizations to manage power and reduce energy consumption.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。