提出首个在线净转化率预测数据集与框架,解决点击后退款延迟带来的建模难题。
Modeling Cascaded Delay Feedback for Online Net Conversion Rate Prediction: Benchmark, Insights and Solutions
- 构建点击-购买-退款三阶段延迟反馈模型,分步解耦转化与退款过程。
- 在淘宝数据集上实现RI-AUC提升12.41%、PRAUC提升14.94%,显著优于现有方法。
- 适用于电商推荐系统优化,尤其适合关注真实用户满意度的工业场景。
在工业级推荐系统中,转化率(CVR)广泛用于流量分配,但因忽略退款行为而无法完整反映推荐效果。为此,净转化率(NetCVR)被提出,定义为点击商品后成功购买且未退款的概率。与传统CVR不同,NetCVR涉及从点击到转化、再从转化到退款的双重延迟反馈,二者影响相反,使传统建模方法失效。此外,缺乏开源数据集与在线持续训练机制也制约了该领域发展。为此,我们推出了CASCADE——首个基于淘宝App的大规模公开数据集,专为在线连续NetCVR预测设计。通过深入分析,我们发现:(1) NetCVR具有强时间动态性,需在线持续建模;(2) 分阶段建模转化率与退款率优于直接建模NetCVR;(3) 延迟时间与转化率及退款率相关,是重要特征。基于此,我们提出TESLA框架,包含转化率-退款率级联结构、分阶段去偏机制及延迟时间感知排序损失。大量实验表明,TESLA在CASCADE上显著优于现有方法,净转化率预测的RI-AUC提升12.41%,RI-PRAUC提升14.94%。代码与数据集已开源。
原文摘要 · Abstract (English)
In industrial recommender systems, conversion rate (CVR) is widely used for traffic allocation, but it fails to fully reflect recommendation effectiveness because it ignores refund behavior. To better capture true user satisfaction and business value, net conversion rate (NetCVR), defined as the probability that a clicked item is purchased and not refunded, has been proposed.Unlike CVR, NetCVR prediction involves a more complex multi-stage cascaded delayed feedback process. The two cascaded delays from click to conversion and from conversion to refund have opposite effects, making traditional CVR modeling methods inapplicable. Moreover, the lack of open-source datasets and online continuous training schemes further hinders progress in this area.To address these challenges, we introduce CASCADE (Cascaded Sequences of Conversion and Delayed Refund), the first large-scale open dataset derived from the Taobao app for online continuous NetCVR prediction. Through an in-depth analysis of CASCADE, we identify three key insights: (1) NetCVR exhibits strong temporal dynamics, necessitating online continuous modeling; (2) cascaded modeling of CVR and refund rate outperforms direct NetCVR modeling; and (3) delay time, which correlates with both CVR and refund rate, is an important feature for NetCVR prediction.Based on these insights, we propose TESLA, a continuous NetCVR modeling framework featuring a CVR-refund-rate cascaded architecture, stage-wise debiasing, and a delay-time-aware ranking loss. Extensive experiments demonstrate that TESLA consistently outperforms state-of-the-art methods on CASCADE, achieving absolute improvements of 12.41 percent in RI-AUC and 14.94 percent in RI-PRAUC on NetCVR prediction. The code and dataset are publicly available at https://github.com/alimama-tech/NetCVR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。