将可信的AI评估方法封装为可扩展的微服务,实现自动监控与保证。
Cloud-Native Evaluation-as-a-Service: A Microservices Architecture for Scalable AI Monitoring with Conformal Guarantees

- 六种无状态微服务组成架构,支持实时评估
- 覆盖率达目标值98.6%以内,漂移检测100%灵敏
- 适合需要高可靠AI监控的工程团队使用
我们提出EaaS,一种云原生参考架构,将六种无状态Kubernetes微服务用于操作化AI评估:有限样本校正的自适应预测集、校准评估、基于RFF近似的最大均值差异漂移检测、带自助置信区间的公平性监控、基于DAG的流水线编排器和结果存储API。验证四项关键问题:第一,在K=50次随机校准/测试划分中,经验覆盖率与名义保证一致,平均偏差小于1.4个百分点;第二,四个MMLU答案标记在前20个logprobs内,无需插补,10%模拟插补时覆盖率影响小于1.5%;第三,RFF-MMD在中位数启发式带宽下对轻微与严重漂移检测率达100%,Ⅰ类错误率为5-8.5%;第四,在UCI Adult Income数据集上,按种族的公平性监控显示显著差异(DP差距=0.33),且跨批次警报稳定。符合性预测与校准服务在批量大小100时,p99延迟低于2毫秒;RFF-MMD需约500毫秒,适用于周期性批量监控。与四种开源工具对比表明,据我们所知,当前尚无平台同时具备符合性预测即服务、微服务分解和DAG编排。
原文摘要 · Abstract (English)
We present EaaS, a cloud-native reference architecture that operationalizes AI evaluation methods as six stateless Kubernetes microservices: conformal prediction with finite-sample-corrected Adaptive Prediction Sets, calibration assessment, drift detection via RFF-approximated Maximum Mean Discrepancy, fairness monitoring with bootstrap confidence intervals, a DAG-based pipeline orchestrator, and a result storage API. We validate four key methodological concerns. First, empirical coverage is consistent with the marginal conformal guarantee across K=50 random calibration/test splits, with mean coverage within 1.4 percentage points of the nominal target. Second, all four MMLU answer tokens appear in the top-20 logprobs with 0% imputation needed, and simulated imputation at 10% produces less than 1.5% coverage impact. Third, RFF-MMD achieves 100% detection power for mild and severe drift at the median heuristic bandwidth, with Type I error between 5-8.5%. Fourth, fairness monitoring on the UCI Adult Income dataset reveals significant demographic parity disparities by race (DP gap=0.33) with stable alerts across sequential batches. Conformal prediction and calibration services achieve sub-2ms p99 latency at batch size 100; RFF-MMD requires ~500ms suited for periodic batch monitoring. A comparison with four open-source tools suggests that, to the best of our knowledge, no current platform combines conformal-prediction-as-a-service, microservice decomposition, and DAG-based orchestration.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。