提出抗干扰多任务梯度提升法,自动识别并抑制异常任务。
Robust-Multi-Task Gradient Boosting
- 分三步:学共性、分出异常任务、精调特异性预测器
- 在真实数据集上显著降低各任务误差,整体性能提升
- 适合处理任务不一致的复杂多任务场景
多任务学习(MTL)通过利用任务间的共享信息提升泛化能力,但现实中常存在与其它任务不匹配的异常或对抗性任务,会损害整体模型性能。为此,我们提出鲁棒多任务梯度提升(R-MTGB),一种新型提升框架,能显式建模并适应任务异质性。该框架包含三个阶段:(1)学习共享模式;(2)通过正则化参数将任务划分为异常与非异常;(3)微调任务特定预测器。此结构使R-MTGB可自动检测并惩罚异常任务,同时促进相关任务间有效知识迁移。方法无缝集成于梯度提升流程中,可在不牺牲精度的前提下稳健应对噪声或对抗性任务。在合成基准和真实数据集上的大量实验表明,该方法成功隔离异常任务,实现知识转移,并持续降低每个任务的预测误差,所有任务整体性能均获提升。结果验证了R-MTGB在复杂多任务环境中的鲁棒性、自适应性与可靠收敛性。
原文摘要 · Abstract (English)
Multi-task learning (MTL) has shown effectiveness in exploiting shared information across tasks to improve generalization. MTL assumes tasks share similarities that can improve performance. In addition, boosting algorithms have demonstrated exceptional performance across diverse learning problems, primarily due to their ability to focus on hard-to-learn instances and iteratively reduce residual errors. This makes them a promising approach for learning multi-task problems. However, real-world MTL scenarios often involve tasks that are not well-aligned (known as outlier or adversarial tasks), which do not share beneficial similarities with others and can, in fact, deteriorate the performance of the overall model. To overcome this challenge, we propose Robust-Multi-Task Gradient Boosting (R-MTGB), a novel boosting framework that explicitly models and adapts to task heterogeneity during training. R-MTGB structures the learning process into three sequential blocks: (1) learning shared patterns, (2) partitioning tasks into outliers and non-outliers with regularized parameters, and (3) fine-tuning task-specific predictors. This architecture enables R-MTGB to automatically detect and penalize outlier tasks while promoting effective knowledge transfer among related tasks. Our method integrates these mechanisms seamlessly within gradient boosting, allowing robust handling of noisy or adversarial tasks without sacrificing accuracy. Extensive experiments on both synthetic benchmarks and real-world datasets demonstrate that our approach successfully isolates outliers, transfers knowledge, and consistently reduces prediction errors for each task individually, and achieves overall performance gains across all tasks. These results highlight robustness, adaptability, and reliable convergence of R-MTGB in challenging MTL environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。