让小模型适配器推理零延迟，性能不降反升
zFLoRA: Zero-Latency Fused Low-Rank Adapters
- 将低秩适配器融合优化，消除推理额外延迟
- 1B~7B模型在18项任务上超越LoRA和全量微调
- 适合移动端和高性能端侧部署的高效适配方案
大型语言模型（LLMs）正越来越多地通过任务特定适配器支持多种下游应用。尽管适配器参数仅占基础模型的1%以下，但推理时带来的额外计算开销却可能高达基础模型的2.5倍。本文提出一种零延迟融合低秩适配器（zFLoRA），在基础模型之上实现零或可忽略的延迟开销。在1B、3B和7B规模的LLM上进行实验，涵盖常识推理、数学推理和摘要对话三类共18个任务，结果表明zFLoRA在性能上优于主流监督微调基准，包括低秩适配器（LoRA）和全量微调（FFT）。在NPU（三星Galaxy S25+）与GPU（NVIDIA H100）平台上测得的延迟显示，zFLoRA适配器引入的延迟为零至可忽略水平。
原文摘要 · Abstract (English)
Large language models (LLMs) are increasingly deployed with task-specific adapters catering to multiple downstream applications. In such a scenario, the additional compute associated with these apparently insignificant number of adapter parameters (typically less than 1% of the base model) turns out to be disproportionately significant during inference time (upto 2.5x times that of the base model). In this paper, we propose a new zero-latency fused low-rank adapter (zFLoRA) that introduces zero or negligible latency overhead on top of the base model. Experimental results on LLMs of size 1B, 3B and 7B show that zFLoRA compares favorably against the popular supervised fine-tuning benchmarks including low-rank adapters (LoRA) as well as full fine-tuning (FFT). Experiments are conducted on 18 different tasks across three different categories namely commonsense reasoning, math reasoning and summary-dialogue. Latency measurements made on NPU (Samsung Galaxy S25+) as well as GPU (NVIDIA H100) platforms show that the proposed zFLoRA adapters introduce zero to negligible latency overhead.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。