用鱼氏信息设计神经网络编码,提升推理效率与容错能力。
Erasure Coded Neural Network Inference via Fisher Averaging
- 基于鱼氏信息构造神经网络编码模型,实现输出线性组合。
- 在真实视觉数据集上,解码准确率显著优于基线方法。
- 适合高延迟容忍的云推理场景,计算开销极低。
擦除编码已在云系统中成功应用,以降低由慢速服务器和异构流量变化引起的尾部延迟。如今,云计算流量中大部分为共享资源上的神经网络推理,其响应时间同样受此类因素影响。然而,现有擦除编码技术主要针对矩阵向量、矩阵矩阵乘法等线性计算,难以适用于高度非线性的神经网络。本文旨在设计一种对神经网络进行编码的方法:给定两个或多个神经网络模型,如何构建一个编码模型,使其输出近似为这些模型输出的线性组合。我们将该问题建模为KL巴雷中心问题,并提出实用算法COIN,利用对角鱼氏信息生成近似实现目标线性组合的编码模型。我们在真实世界视觉数据集上训练的神经网络上进行实验,结果表明,使用COIN解码的输出准确率显著高于其他基线方法,且计算效率极高。
原文摘要 · Abstract (English)
Erasure-coded computing has been successfully used in cloud systems to reduce tail latency caused by factors such as straggling servers and heterogeneous traffic variations. A majority of cloud computing traffic now consists of inference on neural networks on shared resources where the response time of inference queries is also adversely affected by the same factors. However, current erasure coding techniques are largely focused on linear computations such as matrix-vector and matrix-matrix multiplications and hence do not work for the highly non-linear neural network functions. In this paper, we seek to design a method to code over neural networks, that is, given two or more neural network models, how to construct a coded model whose output is a linear combination of the outputs of the given neural networks. We formulate the problem as a KL barycenter problem and propose a practical algorithm COIN that leverages the diagonal Fisher information to create a coded model that approximately outputs the desired linear combination of outputs. We conduct experiments to perform erasure coding over neural networks trained on real-world vision datasets and show that the accuracy of the decoded outputs using COIN is significantly higher than other baselines while being extremely compute-efficient.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。