利用预训练模型预测与梯度参考结合，提升标签稀缺下的优化稳定性。
PPI-SVRG: Unifying Prediction-Powered Inference and Variance Reduction for Semi-Supervised Optimization
- 融合预测值与参考梯度的方差缩减方法
- 理论证明收敛速度受损失几何影响，预测误差仅影响邻域大小
- 适合标签极少但有可靠预训练模型的场景
当标注数据稀缺而预训练模型预测可用时，我们研究半监督随机优化问题。PPI 和 SVRG 均通过控制变量降低方差——PPI 使用预测值，SVRG 使用参考梯度。我们证明二者数学等价，并提出 PPI-SVRG 方法，融合两者优势。其收敛界可分解为标准 SVRG 速率加上由预测不确定性带来的误差下界。收敛速率仅依赖损失几何结构；预测仅影响邻域范围。当预测完美时，完全恢复 SVRG 性能；预测退化时，收敛仍稳定，但达到更大邻域。实验验证理论：在均值估计基准上，标签稀缺下 MSE 降低 43–52%；在仅 10% 标注数据的 MNIST 上，测试准确率提升 2.7–2.9 个百分点。
原文摘要 · Abstract (English)
We study semi-supervised stochastic optimization when labeled data is scarce but predictions from pre-trained models are available. PPI and SVRG both reduce variance through control variates -- PPI uses predictions, SVRG uses reference gradients. We show they are mathematically equivalent and develop PPI-SVRG, which combines both. Our convergence bound decomposes into the standard SVRG rate plus an error floor from prediction uncertainty. The rate depends only on loss geometry; predictions affect only the neighborhood size. When predictions are perfect, we recover SVRG exactly. When predictions degrade, convergence remains stable but reaches a larger neighborhood. Experiments confirm the theory: PPI-SVRG reduces MSE by 43--52\% under label scarcity on mean estimation benchmarks and improves test accuracy by 2.7--2.9 percentage points on MNIST with only 10\% labeled data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。