统一优化多源迁移学习中的权重与迁移量,提升模型性能
Unified Optimization of Source Weights and Transfer Quantities in Multi-Source Transfer Learning: An Asymptotic Framework
- 从渐近分析出发,联合优化各源任务的权重和迁移样本量
- 理论上证明:合理调整权重时,使用全部源数据最优
- 融合费雪信息等要素,适合多源或多任务迁移场景
在多源迁移学习中,如何区分并有效利用异构源任务是一大挑战。现有方法通常只优化源权重或迁移样本量,忽略两者协同。本文提出统一权重与数量优化框架(UOWQ),基于柯尔莫哥洛夫-莱布勒散度的泛化误差渐近分析,将多源迁移建模为参数估计问题。理论发现:当权重合理调整时,使用全部源数据始终最优;最优权重由包含费雪信息、参数差异、维度及迁移量的优化问题决定。基于此,提出实用算法,并扩展至多任务学习场景。在DomainNet和Office-Home等真实基准上,UOWQ持续优于强基线,验证了理论预测与实际有效性。
原文摘要 · Abstract (English)
In multi-source transfer learning, a key challenge lies in how to appropriately differentiate and utilize heterogeneous source tasks. However, existing multi-source methods typically focus on optimizing either the source weights or the amount of transferred samples, largely neglecting their joint consideration. In this work, we propose a theoretical framework, Unified Optimization of Weights and Quantities (UOWQ), that jointly determines the optimal source weights and transfer quantities for each source task. Specifically, the framework formulates multi-source transfer learning as a parameter estimation problem based on an asymptotic analysis of a Kullback--Leibler divergence--based generalization error measure, leading to two main theoretical findings: 1) using all available source samples is always optimal when the weights are properly adjusted; 2) the optimal source weights are characterized by a principled optimization problem whose structure explicitly incorporates the Fisher information, parameter discrepancy, parameter dimensionality, and transfer quantities. Building on the theoretical results, we further propose a practical algorithm for multi-source transfer learning, and extend it to multi-task learning settings where each task simultaneously serves as both a source and a target. Extensive experiments on real-world benchmarks, including DomainNet and Office-Home, demonstrate that UOWQ consistently outperforms strong baselines. The results validate both the theoretical predictions and the practical effectiveness of our framework.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。