arXiv:2604.16400cs.DCcs.AI2026-04

让大模型微调与推理协同运行,提升边缘计算效率。

CoLLM: Continuous Adaptation for SLO-Aware LLM Serving on Shared GPU Clusters

论文配图:CoLLM: Continuous Adaptation for SLO-Aware LLM Serving on Shared GPU Clusters
图 1 · 摘自论文原文
  • 微调与推理共享参数,实时复用模型权重
  • 实测吞吐量最高提升3倍,延迟更低
  • 适合边缘部署需持续优化的智能应用

随着大语言模型在边缘智能中广泛用于特定领域应用与个性化服务,其后训练阶段(包括微调与推理)的质量与效率变得尤为关键,受限于资源条件。尽管联邦参数高效微调(FL PEFT)和低延迟推理近年取得进展,但微调与推理仍被当作独立任务处理,忽视了二者相互依赖性,导致重复部署并延迟推理质量提升。为此,本文提出新型协同执行框架CoLLM,统一共享边缘副本上的FL PEFT与推理任务。CoLLM通过:(1) 副本内模型共享机制,利用未合并推理与影子适配器实现参数实时复用;(2) 两时尺度副本间协调算法,自适应平衡微调与推理负载,联合优化长期模型质量提升与短期推理效率。在多种LLM及真实工作负载上评估显示,CoLLM持续优于现有先进系统,最高实现3倍吞吐量提升,验证了其在边缘智能中实现无缝后训练的能力。

原文摘要 · Abstract (English)

As Large Language Models (LLMs) are increasingly adopted in edge intelligence to power domain-specific applications and personalized services, the quality and efficiency of the LLM post-training phase-including fine-tuning and inference, have become critical due to constrained resources. Although recent advances in federated parameter-efficient fine-tuning (FL PEFT) and low-latency inference have improved individual task performance, fine-tuning and inference are still handled as isolated workloads, which overlooks their interdependence and results in redundant deployments and delayed improvement in inference quality. To address these limitations, we introduce a new co-execution framework and instantiate it with CoLLM, a system that unifies FL PEFT and inference on shared edge replicas and model parameters. CoLLM addresses key challenges at both replica and cluster levels through: (1) an intra-replica model sharing mechanism that enables real-time model parameter reuse via unmerged inference and shadow adapter strategies; and (2) a two-timescale inter-replica coordination algorithm that adaptively balances fine-tuning and inference workloads to jointly optimize long-term model quality gains and short-term inference efficiency. Extensive evaluation across diverse LLMs and real-world traces show that CoLLM consistently outperforms state-of-the-art LLM systems, achieving up to 3x higher goodput, demonstrating its effectiveness in enabling seamless LLM post-training for edge intelligence.

大模型推理边缘计算协同优化参数高效

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。