让模型在增量学习中同时识别物体与状态的组合，避免遗忘。
Not Just Object, But State: Compositional Incremental Learning without Forgetting
- 用提示词机制融合物体与状态信息，解决组合边界模糊问题。
- 在两个数据集上实现当前最优性能，有效识别细粒度状态-物体组合。
- 适合关注细粒度视觉理解与持续学习的研究者。
现有增量学习方法过度关注物体类别，忽视附着于物体的各种状态（如颜色、材质）。为此，我们提出新任务——组合增量学习（composition-IL），使模型能以增量方式识别状态-物体组合的整体特征。由于缺乏合适基准，我们重构两个现有数据集以适配该任务。进而提出基于提示词的组合增量学习器（CompILer），通过多池提示学习并引入池间差异与池内多样性正则化，缓解组合边界模糊问题。此外，采用物体注入状态提示策略，利用物体提示引导状态提示选择，并采用广义均值融合所选提示，消除无关信息。在两个数据集上的大量实验表明，CompILer达到当前最优性能。
原文摘要 · Abstract (English)
Most incremental learners excessively prioritize coarse classes of objects while neglecting various kinds of states (e.g. color and material) attached to the objects. As a result, they are limited in the ability to reason fine-grained compositionality of state-object pairs. To remedy this limitation, we propose a novel task called Compositional Incremental Learning (composition-IL), enabling the model to recognize state-object compositions as a whole in an incremental learning fashion. Since the lack of suitable benchmarks, we re-organize two existing datasets and make them tailored for composition-IL. Then, we propose a prompt-based Composition Incremental Learner (CompILer), to overcome the ambiguous composition boundary problem which challenges composition-IL largely. Specifically, we exploit multi-pool prompt learning, which is regularized by inter-pool prompt discrepancy and intra-pool prompt diversity. Besides, we devise object-injected state prompting by using object prompts to guide the selection of state prompts. Furthermore, we fuse the selected prompts by a generalized-mean strategy, to eliminate irrelevant information learned in the prompts. Extensive experiments on two datasets exhibit state-of-the-art performance achieved by CompILer.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。