Janus让视觉Transformer在动态网络下实现云端边端协同推理，低延迟高精度。
Janus: Collaborative Vision Transformer Under Dynamic Network Environment
- 动态调整模型分割点与剪枝策略，平衡精度与延迟。
- 在不同网络环境下吞吐量提升5.15倍，延迟违规率降低98.7%。
- 适合资源受限设备上部署高性能视觉Transformer的场景。
视觉变换器（ViTs）在各类计算机视觉任务中表现优于传统卷积神经网络，但其计算开销大，要么需压缩后在边缘设备运行，要么需上传原始数据至远程云服务器执行，导致性能下降或延迟升高，限制了广泛应用。本文提出首个面向动态网络环境的低延迟云-边协同视觉变压器推理框架Janus。Janus克服了ViT固有的模型局限，通过结合令牌剪枝与精细到粗略的模型分割策略，以及非静态混合剪枝策略，动态选择最优剪枝级别和分割点，在精度与延迟间取得平衡。实验表明，在多种任务和网络条件下，相比基线方法，Janus将吞吐量最高提升5.15倍，延迟违规率降低最多达98.7%。
原文摘要 · Abstract (English)
Vision Transformers (ViTs) have outperformed traditional Convolutional Neural Network architectures and achieved state-of-the-art results in various computer vision tasks. Since ViTs are computationally expensive, the models either have to be pruned to run on resource-limited edge devices only or have to be executed on remote cloud servers after receiving the raw data transmitted over fluctuating networks. The resulting degraded performance or high latency all hinder their widespread applications. In this paper, we present Janus, the first framework for low-latency cloud-device collaborative Vision Transformer inference over dynamic networks. Janus overcomes the intrinsic model limitations of ViTs and realizes collaboratively executing ViT models on both cloud and edge devices, achieving low latency, high accuracy, and low communication overhead. Specifically, Janus judiciously combines token pruning techniques with a carefully designed fine-to-coarse model splitting policy and non-static mixed pruning policy. It attains a balance between accuracy and latency by dynamically selecting the optimal pruning level and split point. Experimental results across various tasks demonstrate that Janus enhances throughput by up to 5.15 times and reduces latency violation ratios by up to 98.7% when compared with baseline approaches under various network environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。