提出ORBIT方法，防止生成式检索微调时遗忘语言能力。
ORBIT: Preserving Foundational Language Capabilities in GenRetrieval via Origin-Regulated Merging
- 通过监控模型参数距离，动态控制权重平均以限制漂移。
- 在生成式检索任务中保持了92%的原始文本与检索性能。
- 适合需要保留通用语言能力的下游任务微调场景。
尽管大型语言模型（LLM）发展迅速，但在特定任务上微调时常导致其通用语言推理能力发生灾难性遗忘。本文研究了生成式检索（GenRetrieval）任务中的此类遗忘问题。实验发现，遗忘过程迅速且与微调后模型参数与原始模型之间的距离密切相关。基于此，我们提出ORBIT方法，主动追踪微调模型与初始模型权重间的距离，并在距离超过阈值时采用权重平均策略来约束模型漂移。结果表明，ORBIT在保持显著文本与检索性能的同时，优于常见的持续学习基线及采用权重平均的相关正则化方法。
原文摘要 · Abstract (English)
Despite the rapid advancements in large language model (LLM) development, fine-tuning them for specific tasks often results in the catastrophic forgetting of their general, language-based reasoning abilities. This work investigates and addresses this challenge in the context of the Generative Retrieval (GenRetrieval) task. During GenRetrieval fine-tuning, we find this forgetting occurs rapidly and correlates with the distance between the fine-tuned and original model parameters. Given these observations, we propose ORBIT, a novel approach that actively tracks the distance between fine-tuned and initial model weights, and uses a weight averaging strategy to constrain model drift during GenRetrieval fine-tuning when this inter-model distance exceeds a maximum threshold. Our results show that ORBIT retains substantial text and retrieval performance by outperforming both common continual learning baselines and related regularization methods that also employ weight averaging.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。