arXiv:2608.26156cs.AIcs.LG2026-08中稿 · the 15th Internati…

纠正零售长尾数据中的选择偏差,发现分层法比加权法更可靠。

Selection Bias Correction in Retail Intelligence

  • 用分层法和五种加权法对比修正选择偏差,模拟四种数据场景。
  • 分层法在三种场景中误差低于0.04pp,加权法最差时高出116倍。
  • 当选择概率差异大时,加权法失效,分层法更适合作工程方案。

零售智能常依赖高销量热门商品,可能因忽略长尾小众商品而产生经济指标偏差。本仿真研究考察通胀估计中的选择偏差,并比较不同修正方法在四种数据生成过程下的表现。通过400次蒙特卡洛模拟,涵盖对齐阶跃函数、平滑梯度、错位断点及多项式关系四种场景,测试逆概率加权(IPW)五种设定与不同分层数的分层法。结果表明:在长尾零售场景下,加权法存在根本局限——分层法在三个场景中表现更优,即使边界刻意偏离真实断点,其中位误差仍低于0.04pp(比IPW高116倍)。但在光滑多项式关系下,基于样条的IPW表现最佳(中位误差0.007pp vs. 0.013pp),体现情境依赖性。关键发现:即便使用完美结构知识的‘理想’IPW,其在阶跃函数场景中误差仍达6.06pp,远高于分层法的0.008pp,根源在于违反正性假设(Positivity Assumption)——这并非方法缺陷,而是其理论前提被突破。当选择概率悬殊(90% vs. 1%)时,加权法已超出理论适用范围。研究证明,在严重正性缺失的零售长尾分布中,分层法是更安全的工程选择。

原文摘要 · Abstract (English)

Retail intelligence often relies on monitoring popular, high-velocity products, potentially biasing economic indicators by ignoring the "long tail" of niche items. This simulation study investigates selection bias in inflation estimation and compares correction methods across diverse data-generating processes. Through 400 Monte Carlo replications spanning four scenarios--aligned step functions, smooth gradients, misaligned breaks, and polynomial relationships--we test the robustness of Inverse Probability Weighting (IPW) with five specifications against stratification with varying strata counts. Our findings reveal fundamental limits of weighting methods in retail long-tail contexts: stratification achieves superior performance in three of four scenarios, maintaining sub-0.04pp median error even when boundaries deliberately misalign with population breaks (116x advantage over IPW). However, IPW with spline propensity models wins under smooth polynomial relationships (median error 0.007pp vs. 0.013pp), demonstrating context-dependency. Critically, even an oracle IPW specification with perfect structural knowledge achieves 6.06pp error compared to stratification's 0.008pp in step-function scenarios. This reflects violation of the Positivity Assumption--a fundamental causal inference requirement--rather than IPW methodological inferiority. When selection probabilities differ dramatically (90% vs. 1%), weighting methods operate outside their theoretical design envelope. These results demonstrate that stratification provides a safer engineering choice in retail long-tail distributions with severe positivity violations.

选择偏差零售智能因果推断分层法

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。