arXiv:2411.17471cs.LGcs.CR2024-11被引 4

让多模态概念瓶颈模型持续学习新概念旧知识不遗忘

Learning New Concepts, Remembering the Old: Continual Learning for Multimodal Concept Bottleneck Models

  • 将概念与决策层更新转为线性回归,免用梯度优化
  • 实现绝对知识记忆,在新旧概念/类别增量下性能领先
  • 仅靠递归矩阵运算,适合实时大规模多模态应用

概念瓶颈模型(CBMs)提升了AI系统的可解释性,通过连接视觉输入与人类可理解的概念,成为多模态可解释性的重要工具。然而,现有CBMs通常假设数据集静态,难以适应真实世界持续演化的多模态数据流。为此,我们提出一种新型的CBM持续学习任务:同时处理概念增量和类别增量学习。该任务要求模型在持续学习新概念(常代表跨模态属性)和新类别时,稳健保留已有知识。为此,我们提出CONceptual Continual Incremental Learning(CONCIL)框架,将概念与决策层更新重新建模为线性回归问题。此重构避免了基于梯度的优化,有效防止灾难性遗忘。关键的是,CONCIL仅依赖递归矩阵运算,计算高效,非常适合实时与大规模多模态数据应用。实验结果表明,CONCIL实现了‘绝对知识记忆’,在概念增量与类别增量设置下显著优于传统CBM方法,为CBM的持续学习建立了新范式,尤其适用于动态多模态理解场景。

原文摘要 · Abstract (English)

Concept Bottleneck Models (CBMs) enhance the interpretability of AI systems, particularly by bridging visual input with human-understandable concepts, effectively acting as a form of multimodal interpretability model. However, existing CBMs typically assume static datasets, which fundamentally limits their adaptability to real-world, continuously evolving multimodal data streams. To address this, we define a novel continual learning task for CBMs: simultaneously handling concept-incremental and class-incremental learning. This task requires models to continuously acquire new concepts (often representing cross-modal attributes) and classes while robustly preserving previously learned knowledge. To tackle this challenging problem, we propose CONceptual Continual Incremental Learning (CONCIL), a novel framework that fundamentally re-imagines concept and decision layer updates as linear regression problems. This reformulation eliminates the need for gradient-based optimization, thereby effectively preventing catastrophic forgetting. Crucially, CONCIL relies solely on recursive matrix operations, rendering it highly computationally efficient and well-suited for real-time and large-scale multimodal data applications. Experimental results compellingly demonstrate that CONCIL achieves "absolute knowledge memory" and significantly surpasses the performance of traditional CBM methods in both concept- and class-incremental settings, thus establishing a new paradigm for continual learning in CBMs, particularly valuable for dynamic multimodal understanding.

持续学习多模态可解释性概念瓶颈

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。