DRGrad通过动态路由梯度,解决推荐系统多任务学习中的冲突问题。
Direct Routing Gradient (DRGrad): A Personalized Information Surgery for Multi-Task Learning (MTL) Recommendations
- 基于路由与门控机制,动态判断任务间梯度权重
- 在150亿样本数据上显著提升AUC指标,优于现有模型
- 适合处理复杂推荐场景中任务相关性不一的个性化需求
多任务学习(MTL)已成为工业级推荐系统中的一项成功策略,能有效捕捉用户多样兴趣并精准识别点击、停留时长等行为。然而,负迁移与跷跷板现象因真实推荐场景中任务关系复杂且常矛盾而带来挑战。为此,我们提出个性化直接路由梯度框架(DRGrad),包含路由器、更新器与个性化门控网络三个核心组件。DRGrad在训练过程中判断任务间的优先级,可充分利用各任务的有效梯度,降低冲突。我们在包含150亿样本的真实推荐数据集上评估了其性能,结果表明,相比现有先进MTL模型,DRGrad在AUC等指标上表现更优,有效缓解了多任务环境中的任务冲突,且未增加模型复杂度,同时改善了噪声处理能力。此外,在公开的Census-income数据集与合成数据集上的实验也验证了其在不同相关性和个性化程度下准确判断并路由任务间梯度的能力。
原文摘要 · Abstract (English)
Multi-task learning (MTL) has emerged as a successful strategy in industrial-scale recommender systems, offering significant advantages such as capturing diverse users' interests and accurately detecting different behaviors like ``click" or ``dwell time". However, negative transfer and the seesaw phenomenon pose challenges to MTL models due to the complex and often contradictory task correlations in real-world recommendations. To address the problem while making better use of personalized information, we propose a personalized Direct Routing Gradient framework (DRGrad), which consists of three key components: router, updater and personalized gate network. DRGrad judges the stakes between tasks in the training process, which can leverage all valid gradients for the respective task to reduce conflicts. We evaluate the efficiency of DRGrad on complex MTL using a real-world recommendation dataset with 15 billion samples. The results show that DRGrad's superior performance over competing state-of-the-art MTL models, especially in terms of AUC (Area Under the Curve) metrics, indicating that it effectively manages task conflicts in multi-task learning environments without increasing model complexity, while also addressing the deficiencies in noise processing. Moreover, experiments on the public Census-income dataset and Synthetic dataset, have demonstrated the capability of DRGrad in judging and routing the stakes between tasks with varying degrees of correlation and personalization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。