让微调参数随输入动态变化,缓解模型遗忘问题。
Learning When to Adapt
- 引入动态门控机制,使低秩更新根据输入自适应调整。
- 在多个任务上相比LoRA减少遗忘,保持相近微调精度。
- 可解释性强,能定位任务适配集中在哪些层和组件。
低秩适配(LoRA)是一种广泛使用的参数高效微调方法,但其学习到的修正具有静态性:相同的低秩更新应用于所有输入。这种与输入无关的方法在适配微调分布与保留预训练模型在分布外输入上的行为之间产生不可避免的权衡,导致灾难性遗忘。本文提出DISeL(动态输入敏感的LoRA),在LoRA模块中加入轻量级的、针对每个秩-1组件的输入相关门控机制。该门控机制默认保留预训练模型行为,训练时仅激活能降低微调损失的特定组件。DISeL仅增加少量参数,且保持低秩结构。在RoBERTa on GLUE,以及微调用于数学推理和代码生成的Llama和Mistral模型上,DISeL相较于LoRA及其变体减少了遗忘,同时保持了竞争力的微调精度。此外,学习到的门控激活提供了可解释的诊断视角,揭示微调过程中哪些层和秩组件被最频繁激活,从而洞察任务特异性适配的集中位置。代码已公开于 https://github.com/alizindari/DISeL。
原文摘要 · Abstract (English)
Low-rank adaptation (LoRA) is a widely used parameter-efficient fine-tuning method, yet its learned correction is static: the same low-rank update is applied to every input. This input-agnostic approach creates an inevitable compromise between adapting to the fine-tuning distribution and preserving pre-trained behavior on inputs outside that distribution, contributing to catastrophic forgetting. We introduce DISeL (Dynamic Input-Sensitive LoRA), which augments LoRA modules with lightweight input-dependent gates over individual rank-one components. The gating mechanism is designed to preserve the pre-trained model's behavior by default, while training learns to activate selected components that reduce the fine-tuning loss. DISeL adds only a small number of parameters and preserves the low-rank structure. Across RoBERTa on GLUE, and Llama and Mistral models fine-tuned for mathematical reasoning and code generation, DISeL reduces forgetting relative to LoRA and related variants while maintaining competitive fine-tuning accuracy. In addition, the learned gate activations provide an interpretable diagnostic view of which layers and rank components are most activated during fine-tuning, giving insight into where task-specific adaptation is concentrated. Code available at https://github.com/alizindari/DISeL .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。