用持续学习提升零样本动作识别泛化能力
Continual Learning Improves Zero-Shot Action Recognition
- 用合成特征记忆模拟旧类别,融合真实新类别特征进行训练
- 在多个基准上达到零样本识别新纪录,通用零样本设置下性能显著提升
- 适合需要持续学习新动作且不遗忘旧动作的场景
零样本动作识别依赖于从预训练和已见类别向未见新类别的强泛化能力。持续学习则致力于让模型在不遗忘旧任务的前提下有效学习新任务。两者泛化目标高度一致,但持续学习方法尚未被用于零样本识别。本文提出一种基于持续学习的新方法——生成迭代学习(GIL),利用过去类别的合成特征记忆,并将这些合成特征与新类别的真实特征结合。通过该记忆训练分类模型,实现对新旧类别的均衡学习。实验表明,GIL显著提升了未见类别的泛化能力,在多个基准上达到新的零样本识别性能上限。更重要的是,它在更具挑战性的通用零样本设置中表现优异,能有效保留微调前的旧类别知识。
原文摘要 · Abstract (English)
Zero-shot action recognition requires a strong ability to generalize from pre-training and seen classes to novel unseen classes. Similarly, continual learning aims to develop models that can generalize effectively and learn new tasks without forgetting the ones previously learned. The generalization goals of zero-shot and continual learning are closely aligned, however techniques from continual learning have not been applied to zero-shot action recognition. In this paper, we propose a novel method based on continual learning to address zero-shot action recognition. This model, which we call {\em Generative Iterative Learning} (GIL) uses a memory of synthesized features of past classes, and combines these synthetic features with real ones from novel classes. The memory is used to train a classification model, ensuring a balanced exposure to both old and new classes. Experiments demonstrate that {\em GIL} improves generalization in unseen classes, achieving a new state-of-the-art in zero-shot recognition across multiple benchmarks. Importantly, {\em GIL} also boosts performance in the more challenging generalized zero-shot setting, where models need to retain knowledge about classes seen before fine-tuning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。