解决点击率模型训练初期性能骤降问题，提升稳定性和效果
Mitigating Early Training Collapse in CTR Models
- 通过控制特征稀疏度缓解训练崩溃
- 移除高稀疏特征并合并低频值，使训练持续超过1个周期
- 适合工业级点击率预测系统优化
深度神经网络在点击率预测任务中常出现训练首周期后验证性能急剧下降的现象，尽管训练损失仍在持续降低。这种不稳定性阻碍了有效学习，限制了模型表现。本研究基于大规模工业数据集分析该现象，并评估多种缓解策略。结果显示，降低学习率仅带来小幅改善；而控制特征稀疏度则显著提升性能：移除高稀疏特征并聚合低频特征值，可稳定训练过程，使有效学习延续至多个训练周期，同时提升离线评估指标与线上系统表现。
原文摘要 · Abstract (English)
Deep neural models for click-through rate prediction often exhibit a sharp decline in validation performance immediately after the first training epoch despite continued improvement in training loss. This instability restricts effective learning and limits model performance. In this study, we analyze this behavior using large-scale industrial datasets and evaluate practical mitigation strategies. While reducing the learning rate provides only incremental gains, controlling feature sparsity yields substantial improvements. Removing highly sparse features and aggregating infrequent feature values stabilizes training, extends useful learning beyond a single epoch, and improves both offline evaluation metrics and online system performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。