SMART通过谱相似性实现多任务学习,小样本下仍保持高精度。
SMART: A Spectral Transfer Approach to Multi-Task Learning

- 基于源任务的谱结构设计正则化,无需原始数据即可迁移。
- 在小样本下显著提升估计精度,且避免负迁移问题。
- 适合数据受限但存在潜在结构共享的多任务场景。
多任务学习在相关应用中有效,但在目标样本量较小时性能下降。现有迁移学习方法常依赖源-目标模型间有界差异的强假设。本文提出SMART,一种针对多任务线性回归的谱迁移方法,其核心假设为:目标的左右奇异子空间包含于源子空间中,且与源奇异基稀疏对齐。该假设在共享潜在结构时自然成立,可突破有界差异限制。SMART通过融合源任务谱信息的结构化正则化估计目标系数矩阵,仅需已拟合的源模型而非原始数据,适用于数据共享受限场景。尽管优化问题非凸,本文提出基于ADMM的实用算法。建立了通用、非渐近的误差界及无噪声源下的极小极大下界;在附加正则性条件下,达到近乎极小极大的弗罗贝尼乌斯误差率(对数因子内)。模拟实验验证了更高的估计精度和对负迁移的鲁棒性;单细胞多模态数据分析显示更优预测性能。SMART的Python实现及复现代码已公开于https://github.com/boxinz17/smart。
原文摘要 · Abstract (English)
Multi-task learning is effective for related applications, but its performance can deteriorate when the target sample size is small. Transfer learning can borrow strength from related studies; yet, many existing methods rely on restrictive bounded-difference assumptions between the source and target models. We propose SMART, a spectral transfer method for multi-task linear regression that instead assumes spectral similarity: the target left and right singular subspaces lie within the corresponding source subspaces and are sparsely aligned with the source singular bases. Such an assumption is natural when studies share latent structures and enables transfer beyond the bounded-difference settings. SMART estimates the target coefficient matrix through structured regularization that incorporates spectral information from a source study. Importantly, it requires only a fitted source model rather than the raw source data, making it useful when data sharing is limited. Although the optimization problem is nonconvex, we develop a practical ADMM-based algorithm. We establish general, non-asymptotic error bounds and a minimax lower bound in the noiseless-source regime. Under additional regularity conditions, these results yield near-minimax Frobenius error rates up to logarithmic factors. Simulations confirm improved estimation accuracy and robustness to negative transfer, and analysis of multi-modal single-cell data demonstrates better predictive performance. The Python implementation of SMART, along with the code to reproduce all experiments in this paper, is publicly available at https://github.com/boxinz17/smart.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。