评估未部署的观测协议效果,发现数据无法完全确定其价值
Counterfactual Evaluation of Temporal Observation Protocols
- 用反事实框架分析不同观测方案的预测能力
- 有限数据下仍能通过精准测量恢复协议价值识别
- 适合需要优化观测设计的研究者参考
我们研究反事实协议评估:已实施的观测协议所收集的数据能否决定未部署替代方案的预测价值。协议价值定义为从替代方案将采集的测量值中,对固定轨迹目标进行贝叶斯最优预测的总体$R^2$。我们证明,即使拥有无限基准数据,也无法确定该价值:不同的潜在协方差结构可能产生相同的测量-目标分布,却赋予同一替代方案不同价值。我们提出仅关注影响价值的潜在不确定性识别理论。对于线性目标,不可见的协方差方向可导致非识别;而针对性测量可恢复识别,无需重构完整潜变量协方差;精确排列构造将结果扩展至非线性聚合目标。在有限密集校准数据下,统一误差界可控制协议选择遗憾,并区分可辨别的价值差距。精确边际收益支持成本约束、目标感知的观测设计。模拟与睡眠EDF及长期房颤数据的回顾分析表明,广泛的时间布局差异比从有限数据中选择的精细位置更易可靠区分。
原文摘要 · Abstract (English)
We study counterfactual protocol evaluation: whether data collected under a realised observation protocol determine the predictive value of alternatives that were never deployed. Protocol value is the population $R^2$ of the Bayes-optimal predictor of a fixed trajectory-level target from the measurements an alternative would collect. We show that even infinite benchmark data need not determine this value: distinct latent covariance structures can induce the same benchmark measurement--target law while assigning different values to the same alternative. We develop a value-specific identification theory in which only latent ambiguity that changes the alternative's value matters. For linear targets, invisible covariance directions certify non-identification, while targeted measurements can restore identification without recovering the full latent covariance; an exact permutation construction extends the result to nonlinear aggregate targets. With finite dense calibration data, uniform error bounds control protocol-selection regret and distinguishable value gaps. Exact marginal gains then support cost-constrained, target-aware observation design. Simulations and retrospective analyses of Sleep-EDF and Long-Term AF show that broad temporal-layout differences can be more reliably distinguished than fine placements selected from finite data. Together, these results connect identification, calibration resolution and observation design for undeployed protocols.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。