提出实时检测模型风险违规的新方法，应对数据分布突变挑战。
On Continuous Monitoring of Risk Violations under Unknown Shift
- 基于'下注检验'思想设计序贯假设检验流程
- 在多种分布偏移下保持低误报率并及时发现风险违规
- 无需预设偏移类型，适合真实场景持续监控
实际部署的机器学习系统常面临动态且不可预测的数据分布变化，这会破坏事先建立的风险安全保证。现有风险控制框架多依赖固定假设，缺乏持续监控机制。本文提出一种通用框架，用于实时监测演化数据流中的风险违规行为。基于‘测试即下注’范式，设计序贯假设检验方法，可检测与模型决策机制相关的受限风险违规，同时控制误报率。该方法对遭遇的分布偏移类型假设极少，适用性广泛。通过在异常检测和集合预测任务中施加多种分布偏移，验证了方法的有效性。
原文摘要 · Abstract (English)
Machine learning systems deployed in the real world must operate under dynamic and often unpredictable distribution shifts. This challenges the validity of statistical safety assurances on the system's risk established beforehand. Common risk control frameworks rely on fixed assumptions and lack mechanisms to continuously monitor deployment reliability. In this work, we propose a general framework for the real-time monitoring of risk violations in evolving data streams. Leveraging the 'testing by betting' paradigm, we propose a sequential hypothesis testing procedure to detect violations of bounded risks associated with the model's decision-making mechanism, while ensuring control on the false alarm rate. Our method operates under minimal assumptions on the nature of encountered shifts, rendering it broadly applicable. We illustrate the effectiveness of our approach by monitoring risks in outlier detection and set prediction under a variety of shifts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。