用专家混合模型提升少标注数据下的预测精度和可靠性
Prediction-powered Inference by Mixture of Experts
- 将多个预测模型视为专家，动态组合以降低预测方差
- 理论证明其置信区间覆盖误差有上界，优于传统方法
- 适用于均值估计、回归、分位数等任务，适合数据稀缺场景
人工智能领域涌现出多种强大但异构的预测工具，各自具有不同的网络结构、训练策略与领域优势。在标签数据稀缺而无标签数据充足的情况下，这些工具为半监督推断带来新机遇。本文将多个预测器视为专家混合（MOE），构建基于预测驱动推断（PPI）的框架，旨在最小化预测方差。相比标准PPI，该框架能自适应各预测器未知性能，利用集体预测力，并具备最优专家保障。方法适用于均值估计、线性回归、分位数估计及一般M-估计。我们建立了非渐近理论，给出置信区间覆盖误差的上界。数值实验验证了方法的有效性并支持理论结论。
原文摘要 · Abstract (English)
The rapidly expanding artificial intelligence (AI) industry has produced diverse yet powerful prediction tools, each with its own network architecture, training strategy, data-processing pipeline, and domain-specific strengths. These tools create new opportunities for semi-supervised inference, in which labeled data are limited and expensive to obtain, whereas unlabeled data are abundant and widely available. Given a collection of predictors, we treat them as a mixture of experts (MOE) and introduce an MOE-powered semi-supervised inference framework built upon prediction-powered inference (PPI). Motivated by the variance reduction principle underlying PPI, the proposed framework seeks the mixture of experts that achieves the smallest possible variance. Compared with standard PPI, the MOE-powered inference framework adapts to the unknown performance of individual predictors, benefits from their collective predictive power, and enjoys a best-expert guarantee. The framework is flexible and applies to mean estimation, linear regression, quantile estimation, and general M-estimation. We develop non-asymptotic theory for the MOE-powered inference framework and establish upper bounds on the coverage error of the resulting confidence intervals. Numerical experiments demonstrate the practical effectiveness of MOE-powered inference and corroborate our theoretical findings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。