提出无需离散化即可高效检验条件独立性的新方法
A Sample Efficient Conditional Independence Test in the Presence of Discretization
- 基于广义矩估计解决潜变量独立性约束问题
- 在多个数据集上表现优于现有方法，样本效率更高
- 适合处理因测量限制而离散化的实际数据
在许多真实场景中，由于测量限制，关注变量常以离散值表示。直接对这类离散数据应用条件独立（CI）检验可能导致错误结论。尽管近期研究尝试通过二值化观测数据来推断潜变量间的正确CI关系，但该过程不可避免导致信息损失，降低检验性能。本文提出一种不依赖二值化过程的样本高效CI检验方法。我们发现，潜连续变量的独立性可通过解决广义矩估计（GMM）中的过度识别约束问题来建立。基于此，我们推导出合适的检验统计量，并通过节点回归正确确立其渐近分布。理论分析与多种数据集上的实证结果表明，所提方法具有显著优势与有效性。代码已开源：https://github.com/boyangaaaaa/DCT
原文摘要 · Abstract (English)
In many real-world scenarios, interested variables are often represented as discretized values due to measurement limitations. Applying Conditional Independence (CI) tests directly to such discretized data, however, can lead to incorrect conclusions. To address this, recent advancements have sought to infer the correct CI relationship between the latent variables through binarizing observed data. However, this process inevitably results in a loss of information, which degrades the test's performance. Motivated by this, this paper introduces a sample-efficient CI test that does not rely on the binarization process. We find that the independence relationships of latent continuous variables can be established by addressing an over-identifying restriction problem with Generalized Method of Moments (GMM). Based on this insight, we derive an appropriate test statistic and establish its asymptotic distribution correctly reflecting CI by leveraging nodewise regression. Theoretical findings and Empirical results across various datasets demonstrate that the superiority and effectiveness of our proposed test. Our code implementation is provided in https://github.com/boyangaaaaa/DCT
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。