用可学习的非线性距离函数优化复杂损失面，提升小样本元学习收敛速度。
Learnable Loss Geometries with Mirror Descent for Scalable and Convergent Meta-Learning
- 设计可学习的非线性镜映射，替代传统线性预条件，适应复杂损失几何
- 理论证明收敛率可达O(ε⁻²)，实验显示每任务只需少量梯度步即可收敛
- 适合大规模元学习模型，尤其在少样本场景下表现优异
利用相关任务中提取的任务无关知识作为先验信息，元学习为在有限数据下学习新任务提供了一种合理方法。高效地适应该先验信息是元学习面临的主要挑战，因其能仅用少量优化步骤训练出目标任务模型。以往工作通过预条件化加速每任务训练的收敛，但简单线性预条件对复杂损失曲面效果有限。本文提出学习一个灵活的距离生成函数，诱导非线性镜映射以有效捕捉并优化多种损失几何结构。通过合适参数化，该函数由表达能力强的神经网络实现，且可保证为合法距离。理论分析证明所提方法及所有基于预条件的元学习方法均收敛。当梯度范数小于ε时，收敛率为O(ε⁻²)，与标准梯度元学习相当。在少样本学习数据集上的数值实验表明，新算法具有更优的实证性能，且每任务收敛极快，显著减少适配步数，适用于大规模元学习模型。
原文摘要 · Abstract (English)
Utilizing task-invariant knowledge acquired from related tasks as prior information, meta-learning offers a principled approach to learning a new task with limited data records. Sample-efficient adaptation of this prior information is a major challenge facing meta-learning, and plays an important role because it facilitates training the sought task-specific model with just a few optimization steps. Past works deal with this challenge through preconditioning that speeds up convergence of the per-task training. Though effective in representing locally quadratic loss curvatures, simple linear preconditioning can be hardly potent with complex loss geometries. Instead of relying on a quadratic distance metric, the present contribution copes with complex loss metrics by learning a versatile distance-generating function, which induces a nonlinear mirror map to effectively capture and optimize a wide range of loss geometries. With suitable parameterization, this generating function is effected by an expressive neural network that is provably a valid distance. Analytical results establish convergence of not only the proposed method, but also all meta-learning approaches based on preconditioning. To attain gradient norm less than $ε$, the convergence rate of $\mathcal{O}(ε^{-2})$ is on par with standard gradient-based meta-learning methods. Numerical tests on few-shot learning datasets demonstrate the superior empirical performance of the novel algorithm, as well as its rapid per-task convergence, which markedly reduces the number of adaptation steps, hence also accommodating large-scale meta-learning models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。