通过输入变形与仿射变换联合优化,实现低数据场景下高效模型迁移。
Transfer Learning of Surrogate Models: Integrating Domain Warping and Affine Transformations
- 融合输入空间扭曲与仿射变换,适应复杂函数差异
- 仅用少量目标数据点即显著提升模型性能
- 适合数据稀缺的工业场景迁移学习
代理模型为计算成本高昂的真实过程提供高效替代方案,但通常需要大量数据进行有效训练。一种有前景的解决方案是将预训练的代理模型迁移到新任务。以往研究主要关注可微与不可微代理模型的迁移,通常假设源函数与目标函数之间存在仿射变换关系。本文扩展了这一研究,考虑更广泛的变换形式,包括线性与非线性变化。具体地,我们结合未知的输入变形(如由贝塔累积分布函数建模)与未指定的仿射变换,并通过目标任务中少量数据点优化这些变换,以最小化迁移数据集上的经验损失。我们在广泛使用的黑箱优化基准测试平台(BBOB)和汽车行业的实际迁移学习任务上验证了该方法。结果表明,所提方法具有显著优势:迁移后的代理模型在数据稀缺场景下显著优于原始模型及仅用迁移数据从头构建的模型。
原文摘要 · Abstract (English)
Surrogate models provide efficient alternatives to computationally demanding real world processes but often require large datasets for effective training. A promising solution to this limitation is the transfer of pre-trained surrogate models to new tasks. Previous studies have investigated the transfer of differentiable and non-differentiable surrogate models, typically assuming an affine transformation between the source and target functions. This paper extends previous research by addressing a broader range of transformations, including linear and nonlinear variations. Specifically, we consider the combination of an unknown input warping, such as one modeled by the beta cumulative distribution function, with an unspecified affine transformation. Our approach achieves transfer learning by employing a limited number of data points from the target task to optimize these transformations, minimizing empirical loss on the transfer dataset. We validate the proposed method on the widely used Black-Box Optimization Benchmark (BBOB) testbed and a real-world transfer learning task from the automobile industry. The results underscore the significant advantages of the approach, revealing that the transferred surrogate significantly outperforms both the original surrogate and the one built from scratch using the transfer dataset, particularly in data-scarce scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。