用表征相似性指导边缘设备共享神经网络层,无需重训练即可高效压缩模型。
Representation Similarity: A Better Guidance of DNN Layer Sharing for Edge Computing without Training
- 在边缘设备上共享层的输出表征,通过相似性S选择可共享层。
- 表征相似性与模型准确率相关性超0.94,远高于其他指标。
- 无需重训练和真实标签,适合资源受限的实时视频应用。
边缘计算为降低视频流传输延迟和保护隐私提供了新路径。然而,视频应用中日益复杂的深度神经网络(DNNs)给内存受限的边缘设备带来压力。模型合并通过仅保留合并层权重的一份副本,减少内存占用。现有方法存在三方面局限:(i) 仅支持结构相同的层共享;(ii) 需要在云端进行昂贵的重训练;(iii) 依赖真实标签进行重训练。模型性能再评估通常需带真实标签的验证数据集,一般在云端执行。当前常用指导共享层选择的指标包括层大小、计算开销或表征大小。本文提出一种新模型合并方案:在边缘端共享层的输出表征(即表示),并以表征相似性S作为指导。我们证明S与合并后模型准确率的皮尔逊相关系数|r| > 0.94,显著高于其他指标,表明表征相似性可成为无真实标签情况下的强精度预测指标。我们展示了该方案的初步成果及面临挑战,揭示了极具前景的研究方向。
原文摘要 · Abstract (English)
Edge computing has emerged as an alternative to reduce transmission and processing delay and preserve privacy of the video streams. However, the ever-increasing complexity of Deep Neural Networks (DNNs) used in video-based applications (e.g. object detection) exerts pressure on memory-constrained edge devices. Model merging is proposed to reduce the DNNs' memory footprint by keeping only one copy of merged layers' weights in memory. In existing model merging techniques, (i) only architecturally identical layers can be shared; (ii) requires computationally expensive retraining in the cloud; (iii) assumes the availability of ground truth for retraining. The re-evaluation of a merged model's performance, however, requires a validation dataset with ground truth, typically runs at the cloud. Common metrics to guide the selection of shared layers include the size or computational cost of shared layers or representation size. We propose a new model merging scheme by sharing representations (i.e., outputs of layers) at the edge, guided by representation similarity S. We show that S is extremely highly correlated with merged model's accuracy with Pearson Correlation Coefficient |r| > 0.94 than other metrics, demonstrating that representation similarity can serve as a strong validation accuracy indicator without ground truth. We present our preliminary results of the newly proposed model merging scheme with identified challenges, demonstrating a promising research future direction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。