用机器学习优化大模型适配器并发数量，提升显卡吞吐量
A Data-driven ML Approach for Maximizing Performance in LLM-Adapter Serving
- 基于可解释机器学习建模适配器调度策略
- 在真实负载下预测最优并发/并行数误差小于7.2%
- 构建首个可复现推理系统的数字孪生体
随着大语言模型的广泛应用，轻量级适配器成为主流，可在单张GPU上服务数百个适配器以提升吞吐量。然而当显存不足时可能引发请求饥饿问题。本文针对异构适配器与流量特性，提出一种数据驱动的机器学习方法，通过可解释模型优化并发与并行配置，在vLLM框架与LoRA适配器上的实验表明：数字孪生体复现吞吐量误差低于5.1%，机器学习方法在真实负载下对最优并发/并行数的预测误差不超过7.2%。代码已开源。
原文摘要 · Abstract (English)
With the rapid adoption of Large Language Models (LLMs), LLM-adapters have become increasingly common, providing lightweight specialization of large-scale models. Serving hundreds or thousands of these adapters on a single GPU allows request aggregation, increasing throughput, but may also cause request starvation if GPU memory limits are exceeded. To address this issue, this study focuses on determining the joint configuration of concurrent and parallel adapters that maximizes GPU throughput without inducing starvation, given heterogeneous adapter and traffic properties. We propose a data-driven ML approach leveraging interpretable models to tackle this caching problem and introduce the first Digital Twin capable of reproducing an LLM-adapter serving system, enabling efficient training data generation. Experiments with the vLLM framework and LoRA adapters show that the Digital Twin reproduces throughput within 5.1% of real results, while the ML approach predicts optimal numbers of concurrent and parallel adapters with an error of at most 7.2% under heterogeneous, real-world workloads. The code is publicly available at https://github.com/FerranAgulloLopez/GPULLMAdapterOptimization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。