优化无服务器架构下LoRA微调大模型的推理延迟与成本
ServerlessLoRA: Minimizing Latency and Cost in Serverless Inference for LoRA-Based LLMs
- 通过共享基础模型、预加载LoRA参数，减少函数间冗余
- 冷启动延迟降低86%，成本减少89%，实测效果显著
- 适合高并发场景下需要快速响应的LoRA模型部署
无服务器计算因按需付费、细粒度GPU使用和快速扩展，正迅速用于大语言模型（LLM）推理服务。但我们的分析发现，现有无服务器架构在低秩适配（LoRA）推理上存在三大瓶颈：1）函数间存在大量参数冗余，99%的权重被重复存储；2）构建物加载延迟超过LLM本身加载时间；3）多LoRA LLM共存时资源争用加剧。这些导致严重GPU浪费、首令牌生成时间（TTFT）增加及高昂费用。我们提出ServerlessLoRA，一种专为快速低成本运行LoRA LLM设计的无服务器推理系统。该系统支持跨隔离函数的安全基础模型共享以降低冗余，采用预加载策略提前加载完整LoRA构件以减少冷启动延迟，并引入感知争用的批处理与卸载机制，在突发负载下缓解GPU资源冲突。工业级负载实验表明，相比现有先进方案，ServerlessLoRA将TTFT降低最高达86%，成本削减最高达89%。
原文摘要 · Abstract (English)
Serverless computing has grown rapidly for serving Large Language Model (LLM) inference due to its pay-as-you-go pricing, fine-grained GPU usage, and rapid scaling. However, our analysis reveals that current serverless can effectively serve general LLM but fail with Low-Rank Adaptation (LoRA) inference due to three key limitations: 1) massive parameter redundancy among functions where 99% of weights are unnecessarily duplicated, 2) costly artifact loading latency beyond LLM loading, and 3) magnified resource contention when serving multiple LoRA LLMs. These inefficiencies lead to massive GPU wastage, increased Time-To-First-Token (TTFT), and high monetary costs. We propose ServerlessLoRA, a novel serverless inference system designed for faster and cheaper LoRA LLM serving. ServerlessLoRA enables secure backbone LLM sharing across isolated LoRA functions to reduce redundancy. We design a pre-loading method that pre-loads comprehensive LoRA artifacts to minimize cold-start latency. Furthermore, ServerlessLoRA employs contention aware batching and offloading to mitigate GPU resource conflicts during bursty workloads. Experiment on industrial workloads demonstrates that ServerlessLoRA reduces TTFT by up to 86% and cuts monetary costs by up to 89% compared to state-of-the-art LLM inference solutions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。