提出可自动识别共性与特异性特征的迁移学习框架，提升小样本任务性能。
Deep Transfer Learning: Model Framework and Error Analysis
- 自动区分多源数据中的共享与领域特有特征，实现精准迁移。
- 理论证明可显著加快小样本任务收敛速度，最优达O(m⁻¹⁄²)。
- 适合需要可解释性迁移学习的图像分类与回归任务研究者。
本文提出一种深度迁移学习框架，旨在利用多领域上游数据（样本数为n）中丰富的信息，提升单领域下游任务（样本数为m，且m≪n）的性能。该框架允许跨域存在共享与领域特有特征，并支持自动识别，实现信息的精确迁移。同时，明确识别出对下游任务有贡献的上游特征，增强模型可解释性。误差分析表明，该框架可显著加速下游监督任务中Lipschitz函数的学习收敛：从无迁移时的~O(m⁻¹⁄²⁽ᵈ⁺²⁾ + n⁻¹⁄²⁽ᵈ⁺²⁾)，到部分迁移时的~O(m⁻¹⁄²⁽ᵈ⁎⁺³⁾ + n⁻¹⁄²⁽ᵈ⁺²⁾)，甚至完全迁移时的~O(m⁻¹⁄² + n⁻¹⁄²⁽ᵈ⁺²⁾)，其中d*≪d，d为观测数据维度。理论结果在图像分类与回归数据集上得到实证支持。
原文摘要 · Abstract (English)
This paper presents a framework for deep transfer learning, which aims to leverage information from multi-domain upstream data with a large number of samples $n$ to a single-domain downstream task with a considerably smaller number of samples $m$, where $m \ll n$, in order to enhance performance on downstream task. Our framework offers several intriguing features. First, it allows the existence of both shared and domain-specific features across multi-domain data and provides a framework for automatic identification, achieving precise transfer and utilization of information. Second, the framework explicitly identifies upstream features that contribute to downstream tasks, establishing clear relationships between upstream domains and downstream tasks, thereby enhancing interpretability. Error analysis shows that our framework can significantly improve the convergence rate for learning Lipschitz functions in downstream supervised tasks, reducing it from $\tilde{O}(m^{-\frac{1}{2(d+2)}}+n^{-\frac{1}{2(d+2)}})$ ("no transfer") to $\tilde{O}(m^{-\frac{1}{2(d^*+3)}} + n^{-\frac{1}{2(d+2)}})$ ("partial transfer"), and even to $\tilde{O}(m^{-1/2}+n^{-\frac{1}{2(d+2)}})$ ("complete transfer"), where $d^* \ll d$ and $d$ is the dimension of the observed data. Our theoretical findings are supported by empirical experiments on image classification and regression datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。