跨架构高效迁移知识，解决大规模用户预测模型更新难题
Efficient Cross-Architecture Knowledge Transfer for Large-Scale Online User Response Prediction
- 分两阶段设计：离线快速迁移嵌入表，线上动态协同蒸馏
- 实测提升0.27%-0.43% AUC，训练时间减少43%-71%
- 适合需要频繁更新模型的超大规模推荐系统场景
在大规模用户响应预测系统中，部署新架构面临高昂的模型切换成本，源于需对海量历史数据重新训练，且受限于数据保留策略导致性能下降。现有知识蒸馏方法难以应对架构差异，且大尺寸嵌入表迁移成本过高。本文提出CrossAdapt，一种两阶段跨架构知识迁移框架。离线阶段通过维度自适应投影实现快速嵌入迁移，结合渐进式网络蒸馏与策略采样，显著降低计算开销；在线阶段引入非对称协同蒸馏，学生模型高频更新，教师模型低频更新，并采用分布感知适配机制，动态平衡历史知识保留与对演化数据的快速适应。在三个公开数据集上的实验表明，CrossAdapt实现0.27%-0.43% AUC提升，训练时间减少43%-71%。在腾讯微信频道（约1000万日均样本）的大规模部署中，显著缓解了标准蒸馏基线下的AUC下降、LogLoss上升和预测偏差问题。
原文摘要 · Abstract (English)
Deploying new architectures in large-scale user response prediction systems incurs high model switching costs due to expensive retraining on massive historical data and performance degradation under data retention constraints. Existing knowledge distillation methods struggle with architectural heterogeneity and the prohibitive cost of transferring large embedding tables. We propose CrossAdapt, a two-stage framework for efficient cross-architecture knowledge transfer. The offline stage enables rapid embedding transfer via dimension-adaptive projections without iterative training, combined with progressive network distillation and strategic sampling to reduce computational cost. The online stage introduces asymmetric co-distillation, where students update frequently while teachers update infrequently, together with a distribution-aware adaptation mechanism that dynamically balances historical knowledge preservation and fast adaptation to evolving data. Experiments on three public datasets show that CrossAdapt achieves 0.27-0.43% AUC improvements while reducing training time by 43-71%. Large-scale deployment on Tencent WeChat Channels (~10M daily samples) further demonstrates its effectiveness, significantly mitigating AUC degradation, LogLoss increase, and prediction bias compared to standard distillation baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。