提出函数编码器理论，实现三种高效迁移学习方式。
Function Encoders: A Principled Approach to Transfer Learning in Hilbert Spaces
- 基于希尔伯特空间几何构造函数编码器，支持三类迁移。
- 在四个基准上超越现有方法，三类迁移均表现优异。
- 适合研究迁移学习机制或需要快速适应新任务的场景。
迁移学习的核心挑战在于设计能快速适应新任务且无需重训练的算法，但何时何地有效迁移仍缺乏系统刻画。本文提出希尔伯特空间中迁移的几何表征，定义三类归纳迁移：凸包内插、线性张量外推及张量外外推。基于函数编码器理论，提出一种基于最小二乘优化的新训练方案，证明其通用逼近定理，并在四个不同基准上与Transformer和元学习等方法进行全面对比。实验表明，该函数编码器在四类任务中均优于当前最先进方法，在三类迁移策略下均取得最优性能。
原文摘要 · Abstract (English)
A central challenge in transfer learning is designing algorithms that can quickly adapt and generalize to new tasks without retraining. Yet, the conditions of when and how algorithms can effectively transfer to new tasks is poorly characterized. We introduce a geometric characterization of transfer in Hilbert spaces and define three types of inductive transfer: interpolation within the convex hull, extrapolation to the linear span, and extrapolation outside the span. We propose a method grounded in the theory of function encoders to achieve all three types of transfer. Specifically, we introduce a novel training scheme for function encoders using least-squares optimization, prove a universal approximation theorem for function encoders, and provide a comprehensive comparison with existing approaches such as transformers and meta-learning on four diverse benchmarks. Our experiments demonstrate that the function encoder outperforms state-of-the-art methods on four benchmark tasks and on all three types of transfer.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。