针对微小效应的A/B测试，提出更灵敏的带状检测方法。
Strategic A/B testing via Maximum Probability-driven Two-armed Bandit
- 基于反事实框架，用最大概率加权均值波动率设计双臂老虎机策略
- 在零假设下分布更集中，备择假设下更分散，显著提升统计功效
- 适合需要高灵敏度检测微小差异的在线实验场景
在大规模应用中，检测微小的平均处理效应是一个重大挑战，即便极小的改进也可能带来显著的经济影响。传统依赖正态分布或扩展统计量的方法因对微小差异敏感性不足，常无法识别此类效应。本文采用反事实结果框架，提出一种最大概率驱动的双臂老虎机（TAB）过程，通过加权均值波动率统计量控制第一类错误。结合置换方法进一步增强稳健性与有效性。建立的战略中心极限定理（SCLT）表明，该方法在零假设下分布更集中，在备择假设下更分散，极大提升了统计功效。实验结果表明，A/B测试性能显著改善，可在保持高统计功效的同时降低实验成本。
原文摘要 · Abstract (English)
Detecting a minor average treatment effect is a major challenge in large-scale applications, where even minimal improvements can have a significant economic impact. Traditional methods, reliant on normal distribution-based or expanded statistics, often fail to identify such minor effects because of their inability to handle small discrepancies with sufficient sensitivity. This work leverages a counterfactual outcome framework and proposes a maximum probability-driven two-armed bandit (TAB) process by weighting the mean volatility statistic, which controls Type I error. The implementation of permutation methods further enhances the robustness and efficacy. The established strategic central limit theorem (SCLT) demonstrates that our approach yields a more concentrated distribution under the null hypothesis and a less concentrated one under the alternative hypothesis, greatly improving statistical power. The experimental results indicate a significant improvement in the A/B testing, highlighting the potential to reduce experimental costs while maintaining high statistical power.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。