用任务表示实现高效适配器路由，提升大模型多任务适应能力
Effective LoRA Adapter Routing using Task Representations
- 基于任务嵌入而非适配器特征进行路由决策
- 在未见任务上提升5.2分，接近理想性能（101.2%）
- 适合大规模、噪声大的适配器池场景，可扩展至1500+适配器
低秩适配（LoRA）通过模块化适配器实现大语言模型的参数高效定制，催生了涵盖多种任务的公共适配器池。有效使用这些适配器需要路由：为查询选择并组合合适的适配器。我们提出LORAUTER，一种新型路由框架，通过任务表示而非适配器特性来选择和组合LoRA适配器。不同于现有方法直接将查询映射到适配器，LORAUTER利用小规模验证集生成的任务嵌入进行路由，且无需适配器训练数据。在任务层面操作使路由效率随任务数量增长，而非适配器数量。多任务实验表明，LORAUTER始终优于基线方法，在存在任务对齐适配器时达到101.2%的基准性能，并在未见任务上取得领先结果（+5.2分）。我们还验证了LORAUTER在超大规模、高噪声适配器池（超过1500个）下的鲁棒性。
原文摘要 · Abstract (English)
Low-rank adaptation (LoRA) enables parameter efficient specialization of large language models (LLMs) through modular adapters, resulting in rapidly growing public adapter pools spanning diverse tasks. Effectively using these adapters requires routing: selecting and composing the appropriate adapters for a query. We introduce LORAUTER, a novel routing framework that selects and composes LoRA adapters using task representations rather than adapter characteristics. Unlike existing approaches that map queries directly to adapters, LORAUTER routes queries via task embeddings derived from small validation sets and does not require adapter training data. By operating at the task level, LORAUTER achieves efficient routing that scales with the number of tasks rather than the number of adapters. Experiments across multiple tasks show that LORAUTER consistently outperforms baseline routing approaches, matching Oracle performance (101.2%) when task-aligned adapters exist and achieving state-of-the-art results on unseen tasks (+5.2 points). We further demonstrate the robustness of LORAUTER to very large, noisy adapter pools by scaling it to over 1500 adapters.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。