任务间几何差异的分布方式比总量更影响元学习效果
Effects of Structural Allocation of Geometric Task Diversity in Linear Meta-Learning Models
- 区分任务差异中结构相关与无关成分，分析其分配影响
- 当大部分差异落在无关方向时，元学习性能下降
- 适用于研究元学习机制或优化任务设计的研究者
元学习旨在利用相关任务间的共享信息，提升在少量标注数据下的新任务预测能力（少样本学习）。通常认为增加任务多样性可提供更丰富信息，从而提升性能。然而，Kumar 等人（2022）发现，任务表示的整体几何扩展度越高，元学习性能反而可能下降。本文在此基础上指出，元学习表现不仅取决于任务参数的总体几何变异性，更受这种变异在低维结构中的分布方式影响。类似 Pimonova 等人（2025），我们将任务特异性回归效应分解为结构相关信息分量与正交的非信息分量。理论分析与模拟表明，即使整体几何变异性保持不变，若更多变异落入非信息正交方向，元学习预测性能仍会下降。
原文摘要 · Abstract (English)
Meta-learning aims to leverage information across related tasks to improve prediction on unlabeled data for new tasks when only a small number of labeled observations are available ("few-shot" learning). Increased task diversity is often believed to enhance meta-learning by providing richer information across tasks. However, recent work by Kumar et al. (2022) shows that increasing task diversity, quantified through the overall geometric spread of task representations, can in fact degrade meta-learning prediction performance across a range of models and datasets. In this work, we build on this observation by showing that meta-learning performance is affected not only by the overall geometric variability of task parameters, but also by how this variability is allocated relative to an underlying low-dimensional structure. Similar to Pimonova et al. (2025), we decompose task-specific regression effects into a structurally informative component and an orthogonal, non-informative component. We show theoretically and through simulation that meta-learning prediction degrades when a larger fraction of between-task variability lies in orthogonal, non-informative directions, even when the overall geometric variability of tasks is held fixed.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。