解决社交产品实验中的影响扩散问题,提升因果推断准确性。
Towards Reliable Social A/B Testing: Spillover-Contained Clustering with Robust Post-Experiment Analysis
- 构建社交图谱并用平衡聚类算法生成稳定集群
- 新估计器利用行为特征降低方差,提升统计效能
- 适合大规模社交平台的可靠实验设计
A/B测试是在线平台决策的基础,但社交类产品常受网络干扰影响:用户互动导致处理效应溢出至对照组,造成因果估计偏差。现有方法存在局限:用户级随机化忽略网络结构,聚类方法依赖通用聚类,难以兼顾无偏性与统计功效。本文提出两阶段可信赖的社交实验框架。预实验阶段构建社交互动图谱,引入平衡版Louvain算法,在最小化跨簇边数的同时生成大小均衡、稳定的集群,实现可靠的聚类随机化。后实验阶段设计定制化的CUPAC估计器,利用预实验行为协变量降低簇级分配带来的方差,提升统计功效。二者协同实现结构化溢出控制与稳健推断。在日均数亿用户的快手平台进行大规模社交分享实验验证,结果表明该方法显著减少溢出效应,对社交策略评估更准确,建立了可扩展的网络化A/B测试可靠框架。
原文摘要 · Abstract (English)
A/B testing is the foundation of decision-making in online platforms, yet social products often suffer from network interference: user interactions cause treatment effects to spill over into the control group. Such spillovers bias causal estimates and undermine experimental conclusions. Existing approaches face key limitations: user-level randomization ignores network structure, while cluster-based methods often rely on general-purpose clustering that is not tailored for spillover containment and has difficulty balancing unbiasedness and statistical power at scale. We propose a spillover-contained experimentation framework with two stages. In the pre-experiment stage, we build social interaction graphs and introduce a Balanced Louvain algorithm that produces stable, size-balanced clusters while minimizing cross-cluster edges, enabling reliable cluster-based randomization. In the post-experiment stage, we develop a tailored CUPAC estimator that leverages pre-experiment behavioral covariates to reduce the variance induced by cluster-level assignment, thereby improving statistical power. Together, these components provide both structural spillover containment and robust statistical inference. We validate our approach through large-scale social sharing experiments on Kuaishou, a platform serving hundreds of millions of users. Results show that our method substantially reduces spillover and yields more accurate assessments of social strategies than traditional user-level designs, establishing a reliable and scalable framework for networked A/B testing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。