用成本约束优化多个监控器组合，提升AI安全干预召回率。
Combining Cost-Constrained Runtime Monitors for AI Safety
- 基于似然比和奈曼-皮尔逊法则，智能分配监控与干预资源。
- 在代码审查场景中，召回率提升超一倍，优于基线方法。
- 两监控器协同可超越单个监控器，适合高成本敏感场景。
运行时监控有助于发现并阻止有害行为。本文研究如何将多个运行时监控器高效整合为单一监控协议，目标是在平均成本约束下最大化对不对齐输出的安全干预召回率。监控与干预均存在成本，因此协议需在性能与预算间权衡。给定各监控器的性能与成本后，我们提出一种算法，通过穷举搜索确定何时调用哪些监控器，并依据奈曼-皮尔逊引理分配安全干预。该方法聚焦于似然比，战略性地在监控支出与干预支出之间进行权衡。在代码审查任务中，相比朴素基线，召回率提升超过一倍；同时，两个监控器联合使用可实现对单一监控器的帕累托改进。本框架为在成本敏感环境下结合现有监控器检测异常行为提供了严谨的方法论。
原文摘要 · Abstract (English)
Monitoring AIs at runtime can help us detect and stop harmful actions. In this paper, we study how to efficiently combine multiple runtime monitors into a single monitoring protocol. The protocol's objective is to maximize the probability of applying a safety intervention on misaligned outputs (i.e., maximize recall). Since running monitors and applying safety interventions are costly, the protocol also needs to adhere to an average-case budget constraint. Taking the monitors' performance and cost as given, we develop an algorithm to find the best protocol. The algorithm exhaustively searches over when and which monitors to call, and allocates safety interventions based on the Neyman-Pearson lemma. By focusing on likelihood ratios and strategically trading off spending on monitors against spending on interventions, we more than double our recall rate compared to a naive baseline in a code review setting. We also show that combining two monitors can Pareto dominate using either monitor alone. Our framework provides a principled methodology for combining existing monitors to detect undesirable behavior in cost-sensitive settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。