arXiv:2507.17026stat.MLcs.LG2025-07被引 1

弱分类器也能做可靠的两样本检验,只需加个校准步骤。

Conformal C2ST: Turning weak classifiers into strong two-sample tests

  • 用校准方法将任意分类器输出转为精确p值,不依赖最优分类器。
  • 即使分类器表现差,仍能保持稳定检验功效,错误率越低效果越好。
  • 特别适合验证神经后验估计模型的可靠性,实测优于传统方法。

两样本检验是统计与机器学习中的基础任务,旨在判断两组样本是否来自相同分布。经典分类器两样本检验(C2ST)依赖近似贝叶斯最优分类器,但该条件难以满足且难验证。本文证明:弱分类器仍可有效用于检验。基于Hu and Lei (2024)的工作,我们提出两种符合性变体的C2ST,将任意训练好的分类器(即使弱、有偏或过拟合)得分转化为精确的有限样本p值。理论证明其具备两点关键性质:(i) 有限样本下第一类错误率控制;(ii) 检验功效随分类器误差平缓下降。结果表明,即使性能不佳的分类器也能生成强大且可靠的检验。该框架在贝叶斯推断中应用显著,尤其适用于验证神经后验估计(NPE)模型,即将学习的后验近似 $q(θ ext{|}y)$ 与真实后验 $p(θ ext{|}y)$ 的比较转化为两样本检验。实验表明,Conformal C2ST在多个基准测试中优于传统判别性检验。

原文摘要 · Abstract (English)

The two-sample testing problem, a fundamental task in statistics and machine learning, seeks to determine whether two sets of samples, drawn from underlying distributions $p$ and $q$, are in fact identically distributed (i.e. whether $p=q$). A popular and intuitive approach is the classifier two-sample test (C2ST), where a classifier is trained to distinguish between samples from $p$ and $q$. Yet despite simplicity of the C2ST, its reliability hinges on access to a near-Bayes-optimal classifier, a requirement that is rarely met and difficult to verify. This raises a major open question: can a weak classifier still be useful for two-sample testing? We show that the answer is a definitive yes. Building on the work of Hu and Lei (2024), we analyze two conformal variants of the C2ST that convert the scores from any trained classifier -- even if weak, biased, or overfit -- into exact, finite-sample p-values. We establish two key theoretical properties of the conformal C2ST: (i) finite-sample Type-I error control, and (ii) non-trivial power that degrades gently in tandem with the error of the trained classifier. The upshot is that even poorly performing classifiers can yield powerful and reliable two-sample tests. This general framework finds a powerful application in Bayesian inference, particularly for validating Neural Posterior Estimation (NPE) models, where the task of comparing a learned posterior approximation $q(θ\mid y)$ to the true posterior $p(θ\mid y)$ can be framed as a two-sample test. Empirically, the Conformal C2ST outperforms classical discriminative tests across a wide range of benchmarks for this task. Our results establish the conformal C2ST as a practical, theoretically grounded diagnostic tool.

两样本检验校准方法贝叶斯推断NPE

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。