无需训练即可高效调用多个任务专家模型，提升知识密集型任务表现。
LoRA-Augmented Generation (LAG) for Knowledge-Intensive Language Tasks
- 基于分词和层的动态筛选机制，按需调用适配器
- 在无额外数据条件下超越现有方法，在多个任务上表现更优
- 适合需要快速集成领域专家的场景，兼容检索增强生成
针对特定任务和领域的微调语言模型专家日益增多，亟需高效的选取与组合方法。我们提出无需额外训练或数据访问的LoRA-Augmented Generation（LAG），用于利用大规模知识库和任务特定的LoRA适配器。LAG可对每个标记和网络层进行高效过滤、检索与应用专家模型。我们在多种知识密集型任务上评估了LAG，结果表明其性能优于现有无数据方法。同时，我们探讨了有额外数据时的场景，验证了LAG与检索增强生成（RAG）等方案的兼容性。
原文摘要 · Abstract (English)
The proliferation of fine-tuned language model experts for specific tasks and domains signals the need for efficient selection and combination methods. We propose LoRA-Augmented Generation (LAG) for leveraging large libraries of knowledge and task-specific LoRA adapters. LAG requires no additional training or access to data, and efficiently filters, retrieves, and applies experts on a per-token and layer basis. We evaluate LAG on various knowledge-intensive tasks, achieving superior performance over existing data-free methods. We explore scenarios where additional data is available, demonstrating LAG's compatibility with alternative solutions such as retrieval-augmented generation (RAG).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。