无需历史数据的梯度正交化方法，有效缓解多模态大模型持续学习中的遗忘问题。
Octopus: History-Free Gradient Orthogonalization for Continual Learning in Multimodal Large Language Models
- 通过无历史依赖的梯度正交化机制，实现任务间参数隔离。
- 在UCIT数据集上平均性能和最终性能分别领先现有最优6.82%和2.14%。
- 适合需要隐私保护与低存储开销的持续学习场景。
多模态大语言模型的持续学习旨在顺序获取知识的同时缓解灾难性遗忘，但现有方法存在固有局限：基于架构的方法增加计算开销且泛化能力差，基于回放的方法依赖历史数据存储，引发隐私与存储问题，而传统正则化策略不足以完全防止参数干扰。本文提出Octopus，一种基于无历史梯度正交化（HiFGO）的两阶段持续学习框架，无需历史任务数据即可在梯度层面强制正交性。所提出的两阶段微调策略将任务适应与正则化解耦，实现可塑性与稳定性的平衡。在UCIT数据集上的实验表明，Octopus达到当前最优性能，平均准确率与最终准确率分别超越先前SOTA 2.14%和6.82%。
原文摘要 · Abstract (English)
Continual learning in multimodal large language models (MLLMs) aims to sequentially acquire knowledge while mitigating catastrophic forgetting, yet existing methods face inherent limitations: architecture-based approaches incur additional computational overhead and often generalize poorly to new tasks, rehearsal-based methods rely on storing historical data, raising privacy and storage concerns, and conventional regularization-based strategies alone are insufficient to fully prevent parameter interference. We propose Octopus, a two-stage continual learning framework based on History-Free Gradient Orthogonalization (HiFGO), which enforces gradient-level orthogonality without historical task data. Our proposed two-stage finetuning strategy decouples task adaptation from regularization, achieving a principled balance between plasticity and stability. Experiments on UCIT show that Octopus establishes state-of-the-art performance, surpassing prior SOTA by 2.14% and 6.82% in terms of Avg and Last.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。