arXiv:2506.00496cs.AI2025-06KDD被引 4

实时监测黑箱模型输入输出一致性,发现相似输入产生不同结果时报警。

Monitoring Robustness and Individual Fairness

  • 通过固定半径近邻搜索检测相似输入是否导致不同输出。
  • 在标准数据集上验证,能有效识别对抗攻击与个体公平性违规。
  • 支持并行计算,提升大规模部署下的监控效率,适合可信AI系统使用。

输入-输出鲁棒性在文献中以多种形式出现,如模型对对抗性或语义扰动的鲁棒性,以及涉及人类决策的个体公平性。本文提出对已部署的黑箱AI模型进行运行时鲁棒性监测,即观察模型的一段长序列执行过程,一旦发现过去两个相似输入产生了不一致输出,即触发警报。该方法可补充现有的离线鲁棒性增强手段,提升AI决策系统的可信度。我们将监测问题建模为固定半径最近邻(FRNN)搜索,尽管该问题研究广泛,但缺乏合适的在线解决方案。我们开发了Clemont工具,包含多种轻量级监控器,部分采用改进的在线FRNN算法,另一些则基于二进制决策图——一种常用于软硬件验证的数据结构。此外,我们设计了一种高效的并行化技术,显著降低使用L∞范数计算输入-输出对距离的监控器的运行时间。基于对抗鲁棒性和个体公平性领域的标准基准,我们对Clemont中的不同监控器进行了对比实验,证明其在运行时准确检测鲁棒性失效的能力。

原文摘要 · Abstract (English)

Input-output robustness appears in various different forms in the literature, such as robustness of AI models to adversarial or semantic perturbations and individual fairness of AI models that make decisions about humans. We propose runtime monitoring of input-output robustness of deployed, black-box AI models, where the goal is to design monitors that would observe one long execution sequence of the model, and would raise an alarm whenever it is detected that two similar inputs from the past led to dissimilar outputs. This way, monitoring will complement existing offline ``robustification'' approaches to increase the trustworthiness of AI decision-makers. We show that the monitoring problem can be cast as the fixed-radius nearest neighbor (FRNN) search problem, which, despite being well-studied, lacks suitable online solutions. We present our tool Clemont, which offers a number of lightweight monitors, some of which use upgraded online variants of existing FRNN algorithms, and one uses a novel algorithm based on binary decision diagrams -- a data-structure commonly used in software and hardware verification. We have also developed an efficient parallelization technique that can substantially cut down the computation time of monitors for which the distance between input-output pairs is measured using the $L_\infty$ norm. Using standard benchmarks from the literature of adversarial and semantic robustness and individual fairness, we perform a comparative study of different monitors in \tool, and demonstrate their effectiveness in correctly detecting robustness violations at runtime.

AI监控鲁棒性公平性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。