通过干预测试检测预测影响结果的因果效应，提升系统可信度。
Actions Have Consequences: Detecting Outcome Performativity using Intervention Testing
- 用不同预测结果做干预，比较结果分布差异来检测性能化效应。
- 在多个场景下验证了该方法可有效识别性能化效应。
- 适用于样本少、成本高或伦理受限的现实场景，如医疗推荐。
在姑息治疗、信用分配和推荐系统等场景中，预测可能因果性地影响其所预测的结果，这种现象称为结果性能化（Outcome Performativity）。本文提出基于预测干预的检测方法——结果性能化A/B检测（OPAB），通过评估不同预测组别产生的结果分布差异来判断是否存在性能化效应。若差异显著，则判定存在性能化。论文推导了在多种性能化假设类别下的样本复杂度边界，并进行了实证验证。结果显示，OPAB在多数情况下可实现性能化检测；同时发现存在不可区分区域，即在干预次数不足时无法检测。该结果对样本稀缺、成本高昂或伦理敏感场景具有重要实践意义。最后，通过开源带宽数据集案例研究验证了方法有效性，并指明未来方向。
原文摘要 · Abstract (English)
In many domains such as Palliative Care, Credit Assignment and Recommender Systems, predictions may causally influence the outcomes they predict. This phenomena is known as Outcome Performativity. This paper formalises an approach for detecting Outcome Performativity using prediction intervention called Outcome Performativity A/B Detection (OPAB). OPAB enables the detection of Outcome Performativity by assessing the dissimilarity in outcome distributions produced by different predictions groups (interventions). If that dissimilarity is significant, Outcome Performativity is detected. We derive sample complexity bounds for OPAB under various Outcome Performative assumption classes which we empirically validate. Results show that detecting Outcome Performativity using OPAB is achievable in numerous cases. Results also show the presence of regions of indistinguishability which describe settings where the allotted number of interventions are insufficient for detecting Outcome Performativity. The results of which have broader practical implications for the detectability of Outcome Performativity in settings where samples are scarce, cost-prohibitive or potentially unethical to obtain. The paper concludes with a case study on the efficacy of OPAB on the Open Bandits dataset, and provides directions for future work.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。