用逻辑规则检测二分类模型是否出错，无需标签数据。
A logical alarm for misaligned binary classifiers
- 基于二分类结果的共识与分歧建立数学公理体系。
- 仅用无标签数据即可识别至少一个模型失效。
- 适合评估大模型可靠性与安全对齐场景。
当两个智能体在决策上产生分歧时，可怀疑其并非都正确。该论文将这一直觉形式化为对执行二分类任务的智能体进行评估的方法。通过分析它们在联合测试集上的共识与分歧，可推导出唯一一组与响应结果逻辑一致的群体评估方式。此方法基于一组必须被所有二分类响应者共同遵守的公理（代数关系），并针对不同规模的集成（N=1,2）给出了完整的公理系统。利用这些公理，可在不依赖标签数据的情况下构建一个完全逻辑化的异常警报机制，证明至少有一个成员存在功能异常。该方法与形式化软件验证有相似性，对当前安全可信AI的研究具有潜在价值。
原文摘要 · Abstract (English)
If two agents disagree in their decisions, we may suspect they are not both correct. This intuition is formalized for evaluating agents that have carried out a binary classification task. Their agreements and disagreements on a joint test allow us to establish the only group evaluations logically consistent with their responses. This is done by establishing a set of axioms (algebraic relations) that must be universally obeyed by all evaluations of binary responders. A complete set of such axioms are possible for each ensemble of size N. The axioms for $N = 1, 2$ are used to construct a fully logical alarm - one that can prove that at least one ensemble member is malfunctioning using only unlabeled data. The similarities of this approach to formal software verification and its utility for recent agendas of safe guaranteed AI are discussed.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。