构建多模态持续学习工具库，解决模型遗忘与跨模态协同难题
MCITlib: Multimodal Continual Instruction Tuning Library and Benchmark
- 提供8种主流多模态持续学习算法实现
- 在3个基准上验证2个主干模型性能表现
- 适合研究多模态持续学习的学者与工程师使用
持续学习使AI系统能在保留已有知识的前提下不断获取新知识。尽管传统单模态方法已取得进展，但多模态大语言模型（MLLMs）的兴起带来了多模态持续学习（MCL）的新挑战，要求模型同时应对灾难性遗忘和跨模态协调问题。为推动该领域研究，我们提出MCITlib，一个面向多模态持续指令微调的综合性工具库。该库目前实现了8种代表性算法，并在2个主干模型下对3个基准进行了评估。代码库已开源，将持续更新以支持未来MCL发展。
原文摘要 · Abstract (English)
Continual learning enables AI systems to acquire new knowledge while retaining previously learned information. While traditional unimodal methods have made progress, the rise of Multimodal Large Language Models (MLLMs) brings new challenges in Multimodal Continual Learning (MCL), where models are expected to address both catastrophic forgetting and cross-modal coordination. To advance research in this area, we present MCITlib, a comprehensive library for Multimodal Continual Instruction Tuning. MCITlib currently implements 8 representative algorithms and conducts evaluations on 3 benchmarks under 2 backbone models. The library will be continuously updated to support future developments in MCL. The codebase is released at https://github.com/Ghy0501/MCITlib.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。