GLIDE统一了生成式AI评估中的无偏估计方法，显著降低人工标注成本。
Industrializing Prediction-Powered Inference: The GLIDE Library for Reliable GenAI and Agentic Systems Evaluation
- 整合多种预测驱动推断方法与采样策略，实现可靠均值估计。
- 在保持精度前提下，案例研究减少40%以上的人工标注量。
- 适合需要高可信度评估的生成式AI与智能体系统研发人员。
可靠评估智能体系统需要无偏且带有有效置信区间的估计，但当前实践在昂贵的人工标注与有偏的LLM评判代理之间权衡。预测驱动推断（PPI）将两者结合，获得无偏估计与有效置信区间，但其方法分散于多篇论文，实现不完整。本文提出GLIDE，一个开源Python库，统一了最先进的PPI估计器（PPI++、分层PPI、先预测再去偏及其分层变体、主动统计推断）与采样器（均匀、分层、主动、成本最优），采用scipy风格API，专用于均值估计。GLIDE配备可复现的蒙特卡洛验证套件、基于实证的决策树以指导方法选择，并通过智能体评估案例展示在相同精度下大幅减少标注量。该库已开源：https://github.com/EmertonData/glide。
原文摘要 · Abstract (English)
Reliable evaluation of agentic systems requires unbiased estimates with valid uncertainty, but standard practice navigates between costly human annotation and biased LLM-as-judge proxies. Prediction-powered inference (PPI) combines both into debiased estimates with valid confidence intervals, yet its various methods remain scattered across papers under partial implementations. We introduce GLIDE, an open-source Python library that unifies state-of-the-art PPI estimators (PPI++, Stratified PPI, Predict-Then-Debias and its stratified variants, Active Statistical Inference) and samplers (uniform, stratified, active, cost-optimal) under a scipy-style API specialized to mean estimation. GLIDE ships with a reproducible Monte Carlo validation suite, an empirically grounded decision tree for method selection, and an agentic evaluation case study showing substantial annotation savings at equivalent precision. The GLIDE package is available at this URL: https://github.com/EmertonData/glide
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。