提出FoLoRA框架，让大模型微调时不遗忘预训练能力。
Foundation-Preserving Adaptation via Generalized Rayleigh-Quotient Optimization
- 用广义瑞利商筛选更新方向，平衡任务性能与遗忘风险。
- 在数学、代码和指令跟随任务上，保留非目标能力最优。
- 无需固定数据集，通过采样生成预训练代理数据估计遗忘。
微调虽能适配基础模型至下游任务，但可能损害预训练时获得的非目标任务能力。现有防遗忘方法多依赖特殊初始化或固定约束，无法在训练中动态调节适应与保持的权衡。本文提出基础保持型LoRA（FoLoRA），基于一阶保持条件，定义预训练代理激活上的遗忘惩罚与下游任务激活上的任务效用，通过广义瑞利商计算单位遗忘惩罚下的任务效用得分，构建谱坐标系实现方向感知的门控Adam更新，抑制低效用/高遗忘方向。为估算遗忘惩罚，FoLoRA通过从预训练模型采样构造预训练代理校准数据，而非依赖单一代理数据集。在数学、代码和指令遵循任务上的实验表明，相比基线方法，FoLoRA实现了最佳的适应-保持平衡，在提升目标任务性能的同时，最大化非目标能力的整体保留率。
原文摘要 · Abstract (English)
While finetuning effectively adapts foundation models to specialized downstream tasks, it can degrade nontarget capabilities acquired during pretraining. Existing forgetting aware methods typically seek safer updates through specialized initialization or fixed constraints, but do not regulate the adaptation preservation trade-off during training. We propose Foundation Preserving LoRA (FoLoRA), a forgetting aware optimization framework. Guided by a first order preservation condition, FoLoRA defines a forgetting penalty over pretraining-proxy activations and a task utility over downstream task activations. It then scores update directions by task utility per unit forgetting penalty via a generalized Rayleigh quotient. The resulting spectral coordinate system enables direction wise gated Adam updates, attenuating low utility to penalty directions during training. To estimate the forgetting penalty, FoLoRA constructs pretraining proxy calibration data by sampling from the pretrained model rather than relying on a single proxy dataset. Experiments on math, code, and instruction following adaptation show that FoLoRA achieves the strongest preservation adaptation balance over baselines, improving target task performance with best aggregate preservation of non target capabilities.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。