用离线评估加速支付推荐系统优化，省时省钱还准。
Off-policy Evaluation for Payments at Adyen
- 用历史交易数据离线评估推荐算法，替代耗时昂贵的A/B测试。
- OPE结果与线上实验高度一致，预估6个月可多促成9至5400万笔交易。
- 解决高并发下数据方差和扩展性问题，适合大规模工业支付系统。
本文展示了离线策略评估（OPE）在阿登（Adyen）——全球领先的金融支付处理公司——推荐系统研发与优化中的成功应用。面对传统A/B测试耗时长、成本高且常得不出明确结论的局限，团队引入OPE，利用历史数据快速评估新推荐系统版本。分析基于规模达十亿级的交易数据集，结果显示OPE估计值与线上A/B测试结果具有强相关性，预测在六个月内可额外带来900万至5400万笔交易。研究探讨了在高吞吐生产环境中部署OPE所面临的实际挑战与权衡，包括利用探索流量收集数据、降低重要性采样方差，以及通过Apache Spark实现可扩展性。通过对多种OPE估计算法的基准测试，为大型工业支付系统中的决策流程提供有效指导。
原文摘要 · Abstract (English)
This paper demonstrates the successful application of Off-Policy Evaluation (OPE) to accelerate recommender system development and optimization at Adyen, a global leader in financial payment processing. Facing the limitations of traditional A/B testing, which proved slow, costly, and often inconclusive, we integrated OPE to enable rapid evaluation of new recommender system variants using historical data. Our analysis, conducted on a billion-scale dataset of transactions, reveals a strong correlation between OPE estimates and online A/B test results, projecting an incremental 9--54 million transactions over a six-month period. We explore the practical challenges and trade-offs associated with deploying OPE in a high-volume production environment, including leveraging exploration traffic for data collection, mitigating variance in importance sampling, and ensuring scalability through the use of Apache Spark. By benchmarking various OPE estimators, we provide guidance on their effectiveness and integration into the decision-making systems for large-scale industrial payment systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。