arXiv:2605.28809cs.CVcs.LG2026-05中稿 · ICML

针对CLIP的增量学习,分离提取与聚合属性,缓解遗忘问题。

AREA: Attribute Extraction and Aggregation for CLIP-Based Class-Incremental Learning

论文配图:AREA: Attribute Extraction and Aggregation for CLIP-Based Class-Incremental Learning
图 1 · 摘自论文原文
  • 将分类过程拆解为属性提取与聚合,分步优化。
  • 在新类别学习中保持旧类属性稳定,避免灾难性遗忘。
  • 适合需要持续学习的新类别场景,尤其适用于视觉-语言模型。

类增量学习(CIL)对构建真实世界学习系统至关重要。在基于CLIP的CIL中,模型通过比较视觉与文本嵌入的相似性进行分类,例如使用模板提示如“一张[类别]的照片”。这一看似统一的匹配过程可分解为两个概念上独立的阶段:属性提取与属性聚合。例如,识别猫依赖于毛发纹理、胡须等属性;当新增汽车类别时,模型需提取轮子等新属性,并调整其在共享表示空间中的聚合方式。然而,由于仅能访问当前任务数据,增量更新会使属性提取与聚合偏向新类别,导致灾难性遗忘。为此,我们提出AREA方法,用于基于CLIP的属性提取与聚合。为稳定提取,我们通过主测地线分析,将类别级视觉与文本属性锚定于超球面嵌入空间。为稳定聚合,我们引入轻量级任务特定专家,结合评分与残差修正,并以变分信息瓶颈目标正则化。推理时,通过最优传输在任务属性流形间路由,实现更简洁预测。实验表明,AREA持续优于现有最先进方法。代码已公开于 https://github.com/LAMDA-CL/ICML2026-AREA。

原文摘要 · Abstract (English)

Class-Incremental Learning (CIL) is important in building real-world learning systems. In CLIP-based CIL, the model performs classification by comparing similarity between visual and textual embeddings obtained from template prompts, e.g., ``a photo of a [CLASS]''. This seemingly monolithic matching process can be decomposed into two conceptually distinct stages: attribute extraction and attribute aggregation. For example, a model may recognize cat using attributes such as fur texture and whiskers. When learning a new class like car, the model must extract additional attributes like wheels and adjust how they are aggregated in the shared representation space. However, since only data from the current task is available, incremental updates can bias both attribute extraction and aggregation toward new classes, leading to catastrophic forgetting. Therefore, we propose AREA for attribute extraction and aggregation in CLIP-based CIL. To stabilize extraction, we anchor class-level visual and textual attributes on the hyperspherical embedding space via principal geodesic analysis. To stabilize aggregation, we learn lightweight task-specific experts with scoring and residual refinement, regularized by a variational information bottleneck objective. During inference, we perform routing over task attribute manifolds via optimal transport for more concise prediction. Experiments show that AREA consistently outperforms SOTA methods. Code is available at https://github.com/LAMDA-CL/ICML2026-AREA.

增量学习CLIP属性提取视觉语言模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。