用对比学习提升元学习泛化能力，小改动大效果。
Learning to Learn with Contrastive Meta-Objective
- 通过对比模型表示，利用任务身份信息增强元训练
- 在多个元学习模型上实现显著性能提升
- 适配现有方法，适合想提升泛化性的研究者
元学习使学习系统能像人类一样快速适应新任务。现有方法均基于小批量事件训练框架，该框架天然包含任务身份信息，可作为额外监督信号以提升泛化能力。本文受人类快速学习中对齐与区分能力启发，提出将任务身份作为额外监督，通过对比元学习器所学的模型表示来实现。提出的ConML在问题无关和学习者无关的元训练框架下，评估并优化对比元目标。实验表明，ConML可无缝集成至现有元学习器及上下文学习模型，仅需少量实现成本即可带来显著性能提升。
原文摘要 · Abstract (English)
Meta-learning enables learning systems to adapt quickly to new tasks, similar to humans. Different meta-learning approaches all work under/with the mini-batch episodic training framework. Such framework naturally gives the information about task identity, which can serve as additional supervision for meta-training to improve generalizability. We propose to exploit task identity as additional supervision in meta-training, inspired by the alignment and discrimination ability which is is intrinsic in human's fast learning. This is achieved by contrasting what meta-learners learn, i.e., model representations. The proposed ConML is evaluating and optimizing the contrastive meta-objective under a problem- and learner-agnostic meta-training framework. We demonstrate that ConML integrates seamlessly with existing meta-learners, as well as in-context learning models, and brings significant boost in performance with small implementation cost.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。