小数据下用课程学习提升多模态模型表现，尤其适合参数少的模型。
Exploring Curriculum Learning for Vision-Language Tasks: A Study on Small-Scale Multimodal Training
- 按难易程度逐步训练，模拟人类学习过程
- 多模态任务中课程学习提升显著，文本预训练+课程学习效果最好
- 小模型在课程学习下表现更优，适合资源有限场景
在数据和算力受限的特定领域，如何高效训练模型是关键挑战。本文在BabyLM多模态赛道中，研究了三个核心变量在小规模数据下的影响：课程学习、仅文本预训练、模型类型。对比了多模态（文本+图像）与单模态（纯文本）任务的表现。结果表明，在多模态任务中，采用课程学习的模型优于非课程学习模型，尤其当结合文本预训练时效果更佳；而在纯文本任务中，课程学习对参数量较小的模型更有帮助。作者基于模型架构与训练设计差异，提出可能的原因解释。
原文摘要 · Abstract (English)
For specialized domains, there is often not a wealth of data with which to train large machine learning models. In such limited data / compute settings, various methods exist aiming to $\textit{do more with less}$, such as finetuning from a pretrained model, modulating difficulty levels as data are presented to a model (curriculum learning), and considering the role of model type / size. Approaches to efficient $\textit{machine}$ learning also take inspiration from $\textit{human}$ learning by considering use cases where machine learning systems have access to approximately the same number of words experienced by a 13 year old child (100M words). We investigate the role of 3 primary variables in a limited data regime as part of the multimodal track of the BabyLM challenge. We contrast: (i) curriculum learning, (ii), pretraining (with text-only data), (iii) model type. We modulate these variables and assess them on two types of tasks: (a) multimodal (text+image), and (b) unimodal (text-only) tasks. We find that curriculum learning benefits multimodal evaluations over non-curriclum learning models, particularly when combining text-only pretraining. On text-only tasks, curriculum learning appears to help models with smaller trainable parameter counts. We suggest possible reasons based on architectural differences and training designs as to why one might observe such results.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。