用迁移学习提升高维线性回归的泛化能力，实现无需代价的知识迁移。
Transfer Learning for Benign Overfitting in High-Dimensional Linear Regression
- 提出两步迁移MNI方法，利用异构数据源提升目标任务性能。
- 发现免费迁移区间，在特定条件下可零成本获得知识增益。
- 设计数据驱动检测与集成方法，适用于不均衡或异质数据场景。
迁移学习是现代机器学习的核心组件，通过利用多样化数据源来提升目标任务表现。同时，高维线性回归中的最小ℓ₂-范数插值器（MNI）因其出色的泛化能力受到广泛关注，这种现象被称为良性过拟合。尽管二者各自重要，但迁移学习与MNI的交叉研究仍属空白。本文提出一种新型两步迁移MNI方法并分析其权衡关系，刻画了非渐近的额外风险，并识别出其优于仅使用目标数据MNI的条件。分析揭示了‘免费午餐’式协变量偏移情形：在特定条件下，利用异构数据可实现低代价的知识迁移。为实操化该发现，我们开发了一种数据驱动程序以识别信息源，并引入一种融合多个有效迁移MNI的集成方法。有限样本实验表明，所提方法对模型与数据异质性具有鲁棒性，验证了其优势。
原文摘要 · Abstract (English)
Transfer learning is a key component of modern machine learning, enhancing the performance of target tasks by leveraging diverse data sources. Simultaneously, overparameterized models such as the minimum-$\ell_2$-norm interpolator (MNI) in high-dimensional linear regression have garnered significant attention for their remarkable generalization capabilities, a property known as benign overfitting. Despite their individual importance, the intersection of transfer learning and MNI remains largely unexplored. Our research bridges this gap by proposing a novel two-step Transfer MNI approach and analyzing its trade-offs. We characterize its non-asymptotic excess risk and identify conditions under which it outperforms the target-only MNI. Our analysis reveals free-lunch covariate shift regimes, where leveraging heterogeneous data yields the benefit of knowledge transfer at limited cost. To operationalize our findings, we develop a data-driven procedure to detect informative sources and introduce an ensemble method incorporating multiple informative Transfer MNIs. Finite-sample experiments demonstrate the robustness of our methods to model and data heterogeneity, confirming their advantage.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。