用大模型在线生成小模型的LoRA参数，提升特定任务效率与性能。
LoRA-Gen: Specializing Large Language Model via Online LoRA Generation
- 通过重参数化技术，将云端大模型生成的LoRA参数融合到边缘小模型中。
- 在推理任务中实现2.1倍加速，智能代理任务压缩比达10.1倍。
- 无需微调即可超越传统LoRA，适合资源受限的边缘设备部署。
近期研究证明，扩大语言模型规模可显著提升多种自然语言处理任务的表现。然而，在面向特定领域任务时，尤其对小型边缘侧模型而言，现有方法仍存在效果与效率不足的问题。我们提出LoRA-Gen框架，利用云端大型模型根据任务描述生成适用于边缘侧模型的LoRA参数。通过重参数化技术，将生成的LoRA参数合并进边缘模型，实现灵活定制。该方法在保持知识迁移能力的同时，显著提升专用模型的推理效率，通过缩短输入上下文长度实现优化。无需专门训练，LoRA-Gen在推理任务中超越传统LoRA微调，于TinyLLaMA-1.1B上达到2.1倍加速；在智能代理任务中，对Gemma-2B实现10.1倍压缩比。
原文摘要 · Abstract (English)
Recent advances have highlighted the benefits of scaling language models to enhance performance across a wide range of NLP tasks. However, these approaches still face limitations in effectiveness and efficiency when applied to domain-specific tasks, particularly for small edge-side models. We propose the LoRA-Gen framework, which utilizes a large cloud-side model to generate LoRA parameters for edge-side models based on task descriptions. By employing the reparameterization technique, we merge the LoRA parameters into the edge-side model to achieve flexible specialization. Our method facilitates knowledge transfer between models while significantly improving the inference efficiency of the specialized model by reducing the input context length. Without specialized training, LoRA-Gen outperforms conventional LoRA fine-tuning, which achieves competitive accuracy and a 2.1x speedup with TinyLLaMA-1.1B in reasoning tasks. Besides, our method delivers a compression ratio of 10.1x with Gemma-2B on intelligent agent tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。