提出最优样本量理论，高效选择多源迁移学习中的最佳数据量。
A High-Dimensional Statistical Method for Optimizing Transfer Quantities in Multi-Source Transfer Learning
- 基于KL散度与高维统计分析，推导各源任务最优迁移样本量。
- 在多个架构和真实数据集上，准确率与数据效率均超越现有方法。
- 算法不依赖模型结构，适合资源受限场景下的高效迁移学习。
多源迁移学习通过利用多个源任务缓解现实场景中数据稀缺问题。现有方法通常使用所有可用源样本进行训练，限制了训练效率并可能导致次优结果。为此，我们提出一个理论框架，回答‘每个源任务应迁移多少样本以联合训练目标模型’的问题。具体地，我们基于KL散度构建泛化误差度量，并通过高维统计分析最小化该误差，从而确定每个源任务的最优转移样本量。此外，我们设计了一种架构无关且数据高效的算法OTQMS，用于实现上述理论结果。在多种网络架构及两个真实世界基准数据集上的实验表明，所提算法在准确率和数据效率方面显著优于当前最优方法。代码与补充材料见https://github.com/zqy0126/OTQMS。
原文摘要 · Abstract (English)
Multi-source transfer learning provides an effective solution to data scarcity in real-world supervised learning scenarios by leveraging multiple source tasks. In this field, existing works typically use all available samples from sources in training, which constrains their training efficiency and may lead to suboptimal results. To address this, we propose a theoretical framework that answers the question: what is the optimal quantity of source samples needed from each source task to jointly train the target model? Specifically, we introduce a generalization error measure based on K-L divergence, and minimize it based on high-dimensional statistical analysis to determine the optimal transfer quantity for each source task. Additionally, we develop an architecture-agnostic and data-efficient algorithm OTQMS to implement our theoretical results for target model training in multi-source transfer learning. Experimental studies on diverse architectures and two real-world benchmark datasets show that our proposed algorithm significantly outperforms state-of-the-art approaches in both accuracy and data efficiency. The code and supplementary materials are available in https://github.com/zqy0126/OTQMS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。