让大模型在手机上同时完成总结+翻译,速度快还省资源。
On-device System of Compositional Multi-tasking in Large Language Models
- 在适配器上加可学习投影层,实现多任务组合执行
- 手机端实测响应快,比重训或串行处理效率高
- 适合需要低延迟、少资源的移动端复杂任务应用
大型语言模型(LLMs)通常通过参数高效微调技术(如低秩适配器,LoRA)适应下游任务。尽管适配器可组合用于独立处理多任务,但标准方法在同时执行复杂任务(如从长对话生成翻译摘要)时表现不佳。为此,我们提出一种专为组合多任务场景设计的新方法,涉及在总结与翻译适配器组合之上添加可学习投影层。该设计实现了有效整合,同时通过减少计算开销保持高效性,相比需大量重训练或串行处理的替代方案更具优势。我们在Android设备上开发了应用程序,验证了该方法在本地设备环境中的实际可行性。实验结果表明,该方案在云端和设备端均表现良好且快速,凸显其在要求高速运行与资源受限的真实应用场景中的潜力。
原文摘要 · Abstract (English)
Large language models (LLMs) are commonly adapted for diverse downstream tasks via parameter-efficient fine-tuning techniques such as Low-Rank Adapters (LoRA). While adapters can be combined to handle multiple tasks separately, standard approaches struggle when targeting the simultaneous execution of complex tasks, such as generating a translated summary from a long conversation. To address this challenge, we propose a novel approach tailored specifically for compositional multi-tasking scenarios involving summarization and translation. Our technique involves adding a learnable projection layer on top of the combined summarization and translation adapters. This design enables effective integration while maintaining efficiency through reduced computational overhead compared to alternative strategies requiring extensive retraining or sequential processing. We demonstrate the practical viability of our method within an on-device environment by developing an Android app capable of executing compositional tasks seamlessly. Experimental results indicate our solution performs well and is fast in both cloud-based and on-device implementations, highlighting the potential benefits of adopting our framework in real-world applications demanding high-speed operation alongside resource constraints.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。