Loquetier统一了LLM的LoRA微调与推理,提升效率三倍以上。
Loquetier: A Virtualized Multi-LoRA Framework for Unified LLM Fine-tuning and Serving
- 虚拟模块隔离微调参数,支持多适配器共享基础模型
- 融合训练与推理路径,实现最高3倍推理吞吐
- 适合需要高效多任务微调与部署的工程团队
低秩适配(LoRA)已成为适应大语言模型(LLMs)到下游任务的主流参数高效微调(PEFT)技术。尽管已有研究探索了训练与服务的整合,但基于LoRA的微调与推理统一仍存在空白。本文提出Loquetier,一个虚拟化多LoRA框架,将LoRA微调与服务无缝集成于单一运行时。该框架引入两个核心组件:(1) 虚拟模块,隔离基于PEFT的修改并支持多个适配器共享同一基础模型;(2) 优化计算流程与内核设计,合并前向传播中的微调与推理路径,实现高效批处理并减少内核调用开销。在三个任务设置下的实验表明,Loquetier在性能与灵活性上均优于现有基线,在仅推理任务中达到最先进共服务系统3.0倍的吞吐量,在统一微调与推理任务中实现46.4倍更高的SLO达成率。代码已开源:https://github.com/NJUDeepEngine/Loquetier。
原文摘要 · Abstract (English)
Low-Rank Adaptation (LoRA) has become a widely adopted parameter-efficient fine-tuning (PEFT) technique for adapting large language models (LLMs) to downstream tasks. While prior work has explored strategies for integrating LLM training and serving, there still remains a gap in unifying fine-tuning and inference for LoRA-based models. We present Loquetier, a virtualized multi-LoRA framework that seamlessly integrates LoRA fine-tuning and serving within a single runtime. Loquetier introduces two key components: (1) a Virtualized Module that isolates PEFT-based modifications and supports multiple adapters on a shared base model, and (2) an optimized computation flow with a kernel design that merges fine-tuning and inference paths in forward propagation, enabling efficient batching and minimizing kernel invocation overhead. Extensive experiments across three task settings show that Loquetier consistently outperforms existing baselines in both performance and flexibility, achieving up to $3.0\times$ the throughput of the state-of-the-art co-serving system on inference-only tasks and $46.4\times$ higher SLO attainment than PEFT on unified fine-tuning and inference tasks. The implementation of Loquetier is publicly available at https://github.com/NJUDeepEngine/Loquetier.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。