提出新算法Meta-SP，高效学习多任务间共享的低秩不变特征。
Few-shot Multi-Task Learning of Linear Invariant Features with Meta Subspace Pursuit
- 基于多任务线性模型中系数共享低秩分量的假设，设计可证明收敛的元子空间追踪算法。
- 在多种设置下优于ANIL等主流元学习方法，尤其在小样本场景表现更优。
- 适合数据稀缺下的多任务学习，尤其对需跨任务泛化的场景有实用价值。
数据稀缺严重威胁现代机器学习与人工智能的发展，因其实际成功通常依赖大规模数据集。缓解数据不足的一种有效策略是在研究设计阶段利用其他具有相似性的数据源信息，并在分析阶段采用多任务或元学习框架。本文聚焦于多任务（或多元）线性模型，其任务间系数共享一个不变的低秩成分，这是近期多任务或元学习文献中的常见结构假设。在此假设下，我们提出一种新算法——元子空间追踪（Meta Subspace Pursuit，简称Meta-SP），该算法可证明地学习不同任务间共享的不变子空间。在这一简化的多任务/元学习设定下，我们建立了所提方法的算法与统计保证。通过大量数值实验，将Meta-SP与若干竞争方法（包括流行的、无需特定模型的元学习算法ANIL）进行对比，结果表明，Meta-SP在多个方面均显著优于现有方法。
原文摘要 · Abstract (English)
Data scarcity poses a serious threat to modern machine learning and artificial intelligence, as their practical success typically relies on the availability of big datasets. One effective strategy to mitigate the issue of insufficient data is to first harness information from other data sources possessing certain similarities in the study design stage, and then employ the multi-task or meta learning framework in the analysis stage. In this paper, we focus on multi-task (or multi-source) linear models whose coefficients across tasks share an invariant low-rank component, a popular structural assumption considered in the recent multi-task or meta learning literature. Under this assumption, we propose a new algorithm, called Meta Subspace Pursuit (abbreviated as Meta-SP), that provably learns this invariant subspace shared by different tasks. Under this stylized setup for multi-task or meta learning, we establish both the algorithmic and statistical guarantees of the proposed method. Extensive numerical experiments are conducted, comparing Meta-SP against several competing methods, including popular, off-the-shelf model-agnostic meta learning algorithms such as ANIL. These experiments demonstrate that Meta-SP achieves superior performance over the competing methods in various aspects.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。