用YouTube大模型零样本迁移，提升低流量音乐推荐效果
Zero-shot Cross-domain Knowledge Distillation: A Case study on YouTube Music
- 从高数据量的YouTube视频推荐中，零样本迁移知识到低流量音乐应用
- 在音乐推荐模型上实现显著性能提升，验证了跨域蒸馏的有效性
- 适合资源有限、需快速优化推荐系统的团队参考
知识蒸馏（KD）广泛用于提升服务实时流量的低延迟模型质量。但在低流量生产推荐系统中应用时面临挑战：数据量有限限制教师模型规模，训练大型专用教师成本过高。跨域知识蒸馏通过利用数据丰富的源域教师模型提供低成本替代方案，但因特征、用户界面和预测任务差异带来独特技术难题。本文针对多任务排序模型，开展零样本跨域知识蒸馏的案例研究，将100倍规模的YouTube视频推荐平台作为教师，迁移到低流量音乐推荐应用。通过离线与线上实验，评估了多种蒸馏方法在该场景下的表现，结果表明零样本跨域蒸馏是提升低流量场景下排序模型性能的实用且有效方法。
原文摘要 · Abstract (English)
Knowledge Distillation (KD) has been widely used to improve the quality of latency sensitive models serving live traffic. However, applying KD in production recommender systems with low traffic is challenging: the limited amount of data restricts the teacher model size, and the cost of training a large dedicated teacher may not be justified. Cross-domain KD offers a cost-effective alternative by leveraging a teacher from a data-rich source domain, but introduces unique technical difficulties, as the features, user interfaces, and prediction tasks can significantly differ. We present a case study of using zero-shot cross-domain KD for multi-task ranking models, transferring knowledge from a (100x) large-scale video recommendation platform (YouTube) to a music recommendation application with significantly lower traffic. We share offline and live experiment results and present findings evaluating different KD techniques in this setting across two ranking models on the music app. Our results demonstrate that zero-shot cross-domain KD is a practical and effective approach to improve the performance of ranking models on low traffic surfaces.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。