首个评估大模型违规行为监测可靠性的基准，发现安全与实用难兼顾。
AutoMonitor-Bench: Evaluating the Reliability of LLM-Based Misbehavior Monitor
- 构建3010个带标注样本的基准，覆盖问答、代码生成等任务
- 12款私有与10款开源模型表现差异大，误检率与漏检率呈权衡关系
- 训练数据扩展至15万+样本，仍难应对隐蔽违规行为，适合安全研究者
我们提出AutoMonitor-Bench，首个系统评估大模型驱动的违规行为监测器可靠性的基准。该基准包含3,010个精心标注的测试样本，覆盖问答、代码生成和推理任务，每对样本均包含违规与正常实例。采用漏检率（MR）和误报率（FAR）两个互补指标，分别衡量未能检测违规行为和对正常行为过度敏感的问题。评估12款私有及10款开源大模型后，发现监控性能存在显著差异，且MR与FAR间存在持续权衡，揭示了安全与可用性之间的内在矛盾。为进一步探索监测可靠性极限，我们构建了包含153,581个样本的大规模训练语料库，并微调Qwen3-4B-Instruction模型，验证在已知易构造的违规数据集上训练是否能提升对未知、更隐晦违规行为的检测能力。结果表明，可靠、可扩展的违规监测仍面临挑战，亟需面向任务设计和训练策略的创新。
原文摘要 · Abstract (English)
We introduce AutoMonitor-Bench, the first benchmark designed to systematically evaluate the reliability of LLM-based misbehavior monitors across diverse tasks and failure modes. AutoMonitor-Bench consists of 3,010 carefully annotated test samples spanning question answering, code generation, and reasoning, with paired misbehavior and benign instances. We evaluate monitors using two complementary metrics: Miss Rate (MR) and False Alarm Rate (FAR), capturing failures to detect misbehavior and oversensitivity to benign behavior, respectively. Evaluating 12 proprietary and 10 open-source LLMs, we observe substantial variability in monitoring performance and a consistent trade-off between MR and FAR, revealing an inherent safety-utility tension. To further explore the limits of monitor reliability, we construct a large-scale training corpus of 153,581 samples and fine-tune Qwen3-4B-Instruction to investigate whether training on known, relatively easy-to-construct misbehavior datasets improves monitoring performance on unseen and more implicit misbehaviors. Our results highlight the challenges of reliable, scalable misbehavior monitoring and motivate future work on task-aware designing and training strategies for LLM-based monitors.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。