测试大模型在错误对话中是否能拒绝无关请求，发现所有模型都严重不安全。
OffTopicEval: When Large Language Models Enter the Wrong Chat, Almost Always!
- 设计新评测基准，衡量模型在特定任务中拒绝无关请求的能力。
- 20个开源模型均表现不佳，最强仅79.96%正确拒绝，多数低于50%。
- 用提示词引导方法可提升拒绝率，最高增41%，适合企业部署前评估。
大型语言模型（LLM）安全性是其大规模应用的关键挑战。现有研究多关注通用危害，但企业更关心模型在特定用途下的操作安全性——即能否正确接受或拒绝与任务无关的请求。为此，本文提出操作安全性概念，并构建了OffTopicEval评测套件，用于评估通用及特定代理场景下的操作安全性。对六类模型共20个开源LLM的测试显示，性能虽有差异，但所有模型均存在严重不安全问题：最强模型Qwen-3（235B）仅为77.77%，Mistral（24B）为79.96%，而GPT系列稳定在62%-73%，Phi为48%-70%，Gemma和Llama-3分别低至39.53%和23.84%。为缓解此问题，提出两种基于提示词的引导方法：查询锚定（Q-ground）和系统提示锚定（P-ground），显著提升模型在非预期场景下的拒绝能力。其中，Q-ground提升最高达23%，P-ground效果更强，使Llama-3.3（70B）提升41%，Qwen-3（30B）提升27%。结果表明，亟需加强操作安全性干预，而提示词引导为迈向更可靠代理的第一步。
原文摘要 · Abstract (English)
Large Language Model (LLM) safety is one of the most pressing challenges for enabling wide-scale deployment. While most studies and global discussions focus on generic harms, such as models assisting users in harming themselves or others, enterprises face a more fundamental concern: whether LLM-based agents are safe for their intended use case. To address this, we introduce operational safety, defined as an LLM's ability to appropriately accept or refuse user queries when tasked with a specific purpose. We further propose OffTopicEval, an evaluation suite and benchmark for measuring operational safety both in general and within specific agentic use cases. Our evaluations on six model families comprising 20 open-weight LLMs reveal that while performance varies across models, all of them remain highly operationally unsafe. Even the strongest models - Qwen-3 (235B) with 77.77% and Mistral (24B) with 79.96% - fall far short of reliable operational safety, while GPT models plateau in the 62-73% range, Phi achieves only mid-level scores (48-70%), and Gemma and Llama-3 collapse to 39.53% and 23.84%, respectively. While operational safety is a core model alignment issue, to suppress these failures, we propose prompt-based steering methods: query grounding (Q-ground) and system-prompt grounding (P-ground), which substantially improve OOD refusal. Q-ground provides consistent gains of up to 23%, while P-ground delivers even larger boosts, raising Llama-3.3 (70B) by 41% and Qwen-3 (30B) by 27%. These results highlight both the urgent need for operational safety interventions and the promise of prompt-based steering as a first step toward more reliable LLM-based agents.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。