通过可组合的子空间编辑，让大模型多任务学习不互相干扰。
Compositional Subspace Representation Fine-tuning for Adaptive Large Language Models
- 在隐藏层表示中学习正交子空间，分别对应不同技能。
- 在AlpacaEval上达到93.94%胜率，仅用0.0098%参数量。
- 适合需要轻量多任务适配的场景，如边缘设备部署。
将大语言模型适配多个任务时会出现跨技能干扰，即提升一项技能会损害另一项。尽管LoRA等方法在权重层面施加正交约束，但未能完全解决隐藏状态表示中的干扰问题。本文提出基于表示的新型方法CS-ReFT，学习多个正交子空间变换，每个专精于特定技能，并通过轻量路由机制进行组合。通过将子空间修改隔离在隐藏状态中而非权重矩阵，CS-ReFT更有效地避免跨任务冲突。在AlpacaEval基准测试中，对Llama-2-7B应用CS-ReFT后取得93.94%胜率，超过GPT-3.5 Turbo（86.30%），且仅需0.0098%的模型参数。结果表明，通过简单路由组合的专用表示编辑，能显著提升多任务指令遵循能力，且开销极低。
原文摘要 · Abstract (English)
Adapting large language models to multiple tasks can cause cross-skill interference, where improvements for one skill degrade another. While methods such as LoRA impose orthogonality constraints at the weight level, they do not fully address interference in hidden-state representations. We propose Compositional Subspace Representation Fine-tuning (CS-ReFT), a novel representation-based approach that learns multiple orthonormal subspace transformations, each specializing in a distinct skill, and composes them via a lightweight router. By isolating these subspace edits in the hidden state, rather than weight matrices, CS-ReFT prevents cross-task conflicts more effectively. On the AlpacaEval benchmark, applying CS-ReFT to Llama-2-7B achieves a 93.94% win rate, surpassing GPT-3.5 Turbo (86.30%) while requiring only 0.0098% of model parameters. These findings show that specialized representation edits, composed via a simple router, significantly enhance multi-task instruction following with minimal overhead.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。