arXiv:2511.11608cs.DCcs.AI2025-11

不重训练,压缩中间特征,让边缘设备多干活,服务器更省力。

Why Should the Server Do It All?: A Scalable, Versatile, and Model-Agnostic Framework for Server-Light DNN Inference over Massively Distributed Clients via Training-Free Intermediate Feature Compression

  • 用稀疏过滤+分块量化压缩中间特征,无需重训练
  • 通信量减少10倍,服务器耗时降低4.4倍,精度损失仅0-3个百分点
  • 适合大规模分布式推理,尤其适配自回归大模型

现代深度神经网络常采用边缘-云模型分割(MP),但现有方案固定浅层分割点,导致边缘算力浪费,且将延迟与能耗集中于服务器。这一问题在自回归大模型(AR LLM)推理中尤为严重,因每轮生成需重复计算大量中间特征(IFs)。我们提出SLICER:一种无需重训练、架构无关的框架,通过压缩中间特征来降低通信开销和服务器负载。SLICER结合三种技术:(i) 非对称Top-K过滤(ATKF)剔除低幅值激活;(ii) 幅值拆分(MS)将剩余非零值均分至等卡数块;(iii) 自适应比特量化(ABQ)在失真预算下为每块选择最优位宽。在标准视觉与语言模型任务(如ImageNet/COCO;HellaSwag, PIQA, ARC-E/C, GSM8K, HumanEval)上,SLICER可使上行流量减少最高10倍,服务器GPU时间减少最高4.4倍,任务性能保持在基线~0-3个百分点内。在多设备与自回归大模型场景下,该框架通过将计算前移至边缘,降低每令牌比特数与服务器时间,稳定每步流量。该编码器可直接附加至现成模型,无需重训练或结构修改,提供即插即用的可扩展低延迟分布式推理路径。代码见附录。

原文摘要 · Abstract (English)

Modern DNNs often rely on edge-cloud model partitioning (MP), but widely used schemes fix shallow, static split points that underutilize edge compute and concentrate latency and energy on the server. The problem is exacerbated in autoregressive (AR) LLM inference, where per-token forward passes repeatedly generate bulky intermediate features (IFs). We introduce SLICER, a retraining-free, architecture-agnostic framework that compresses IFs to reduce both communication and server load in split computing. SLICER combines (i) asymmetric top-K filtering (ATKF) to sparsify low-magnitude activations, (ii) magnitude-splitting (MS) to group the remaining non-zeros into equal-cardinality blocks, and (iii) adaptive bit quantization (ABQ) that selects per-block bitwidths under a distortion budget. Across standard vision and LLM workloads (e.g., ImageNet/COCO; HellaSwag, PIQA, ARC-E/C, GSM8K, HumanEval), SLICER reduces uplink volume by up to 10x and server GPU time by up to 4.4x, while keeping task quality within ~0-3 pp of baseline. In multi-device settings and AR LLMs, SLICER scales by shifting meaningful compute to the edge and lowering bits-per-token and server time per token, stabilizing per-step traffic. The codec attaches to off-the-shelf models without retraining or architectural changes, offering a plug-and-play path to scalable, low-latency distributed inference. Code is provided in the supplementary material.

模型分割边缘计算压缩推理大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。