arXiv:2607.02681stat.MLcs.LG2026-07

提出新方法,在任务污染与异质性下同时实现鲁棒性和个性化估计。

Contaminated Multi-task Learning with Heterogeneity: Fundamental Limits and Optimal Algorithms

  • 基于过滤机制的梯度下降法,有效识别并剔除污染任务。
  • 理论证明其误差逼近最优率,避免传统方法的维度依赖缺陷。
  • 适合存在异常数据且任务差异大的联邦学习、多任务学习场景。

在包含 K 个任务、每任务样本量为 n 的污染多任务经验风险最小化框架中,ε 分数的任务可能被任意污染,其余任务具有异质性。目标是同时估计全局平均风险最小值和干净的任务特定最小值,兼顾鲁棒性与个性化。在高斯均值模型下,发现自适应正则、全局矩阵正则、分解正则及基于得分的异常任务检测等常见方法的最坏情况污染误差为 ε√(d/n),低于最优下界 ε/√n,揭示了维度相关的障碍。进一步建立了广义异质性 ERM 的极小极大下界,并提出一种计算高效的基于过滤的鲁棒多任务梯度下降法。在局部强凸性、光滑性和次高斯梯度假设下,该方法在高概率下达到接近极小极大率的上界,去除许多正则化方法和得分检测的 √d 依赖,且在强异质性下仍能实现个性化。模拟与真实数据实验表明,其相比多种基准方法展现出更强的鲁棒性与个性化能力。

原文摘要 · Abstract (English)

Integrating information across related tasks can improve estimation and prediction in transfer, multi-task, and federated learning, but contamination and heterogeneity make robust borrowing challenging. We study a contaminated multi-task empirical risk minimization (ERM) framework in which an $ε$ fraction of $K$ tasks, each with sample size $n$, may be arbitrarily contaminated while the remaining tasks are heterogeneous. Our goal is to estimate both the global minimizer of the average risk and the clean task-specific minimizers, thereby combining robustness and personalization. In the Gaussian mean model, we show that several common paradigms, including adaptive and robust regularization around a shared center, global matrix regularization, decomposition-based regularization, and score-based outlier-task detection, all suffer from a worst-case contamination error of order $ε\sqrt{d/n}$, which is suboptimal compared to the lower bound $ε/\sqrt{n}$. This identifies a dimension-dependent barrier for these approaches. We then establish minimax lower bounds for a general heterogeneous ERM setting and propose a computationally efficient filtering-based robust multi-task gradient descent method. Under local strong convexity, smoothness, and sub-Gaussian gradient assumptions, the proposed method attains high-probability upper bounds matching the minimax rates up to logarithmic factors over a broad regime. In particular, it removes the extra $\sqrt{d}$ contamination dependence of many regularization-based methods and score-based outlier detection, while achieving personalization to local tasks under strong heterogeneity. Simulations and a real-data analysis demonstrate strong robustness and personalization relative to a broad range of benchmark methods.

多任务学习鲁棒性联邦学习优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。