arXiv:2601.21873cs.LGstat.ML2026-01

在维度增长下,用低秩加稀疏结构实现跨任务参数迁移学习。

Low-Rank Plus Sparse Matrix Transfer Learning under Growing Representations and Ambient Dimensions

  • 目标参数分解为源任务嵌入、低秩创新和稀疏修改三部分。
  • 理论证明在秩与稀疏度增量小时,误差率严格优于不迁移。
  • 适用于马尔可夫转移矩阵和协方差估计等典型场景。

学习系统常随时间扩展其环境特征或潜在表示,将早期表示嵌入更大的空间但新增的潜在结构有限。本文研究在环境维度与内在表示同步增长下的结构化矩阵估计迁移学习问题,其中已良好估计的源任务被嵌入高维目标任务的子空间中。提出一种通用迁移框架:目标参数由嵌入的源成分、低维低秩创新项和稀疏修正项组成;并设计锚定交替投影估计器,在保持转移子空间的同时仅估计低维创新与稀疏修改。建立确定性误差界,将目标噪声、表示增长与源估计误差分离,当秩与稀疏度增量较小时,获得严格更优的收敛速率。通过两个典型问题验证框架普适性:对单轨迹马尔可夫转移矩阵估计,给出依赖噪声下的端到端理论保证;对维度扩大的结构化协方差估计,附录提供补充理论分析,并通过实证验证一致的迁移增益。

原文摘要 · Abstract (English)

Learning systems often expand their ambient features or latent representations over time, embedding earlier representations into larger spaces with limited new latent structure. We study transfer learning for structured matrix estimation under simultaneous growth of the ambient dimension and the intrinsic representation, where a well-estimated source task is embedded as a subspace of a higher-dimensional target task. We propose a general transfer framework in which the target parameter decomposes into an embedded source component, low-dimensional low-rank innovations, and sparse edits, and develop an anchored alternating projection estimator that preserves transferred subspaces while estimating only low-dimensional innovations and sparse modifications. We establish deterministic error bounds that separate target noise, representation growth, and source estimation error, yielding strictly improved rates when rank and sparsity increments are small. We demonstrate the generality of the framework by applying it to two canonical problems. For Markov transition matrix estimation from a single trajectory, we derive end-to-end theoretical guarantees under dependent noise. For structured covariance estimation under enlarged dimensions, we provide complementary theoretical analysis in the appendix and empirically validate consistent transfer gains.

迁移学习低秩模型稀疏结构高维统计

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。