无需模型结构相似性,用目标数据增强源域表征实现高效迁移学习。
Transfer Learning through Enhanced Sufficient Representation: Enriching Source Domain Knowledge with Target Data
- 从源域提取不变表征,再融合目标域独立成分进行增强。
- 可在源域回归、目标域分类等异构任务间迁移,适用范围广。
- 理论与实证均证明其在小样本下有效,突破传统方法限制。
迁移学习是应对各类应用中数据稀缺问题的重要方法,通过将成熟源域的知识迁移到不熟悉的靶域来实现。然而,传统方法常受限于僵化的模型假设及源域与靶域模型高度相似的要求。本文提出一种名为通过增强充分表征的迁移学习(TESR)的新方法:首先从源域估计一个充分且不变的表征,再引入来自靶域的独立分量进行增强,确保该表征对靶域充分且能适应其特性。特斯的优势在于不依赖不同任务间模型结构的相似性——例如源域可为回归模型,靶域可为分类任务。此灵活性使特斯适用于多种监督学习场景。我们分析了特斯的理论性质,并通过模拟研究和真实数据应用验证其性能,结果表明其在有限样本条件下仍具有效性。
原文摘要 · Abstract (English)
Transfer learning is an important approach for addressing the challenges posed by limited data availability in various applications. It accomplishes this by transferring knowledge from well-established source domains to a less familiar target domain. However, traditional transfer learning methods often face difficulties due to rigid model assumptions and the need for a high degree of similarity between source and target domain models. In this paper, we introduce a novel method for transfer learning called Transfer learning through Enhanced Sufficient Representation (TESR). Our approach begins by estimating a sufficient and invariant representation from the source domains. This representation is then enhanced with an independent component derived from the target data, ensuring that it is sufficient for the target domain and adaptable to its specific characteristics. A notable advantage of TESR is that it does not rely on assuming similar model structures across different tasks. For example, the source domain models can be regression models, while the target domain task can be classification. This flexibility makes TESR applicable to a wide range of supervised learning problems. We explore the theoretical properties of TESR and validate its performance through simulation studies and real-world data applications, demonstrating its effectiveness in finite sample settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。