测试53个模型在11个数据集上的安全表现，发现没有一个模型能应对所有风险。
No One Model Catches Every Harm: Benchmarking Content Moderation Across Safety Scenarios
- 系统评估53个模型在11个数据集上的内容安全能力，分四类场景测试
- 大模型在某些场景领先，但在其他场景被小而专的模型超越
- 对话场景的安全问题普遍未解决，规模不等于安全
大型语言模型（LLMs）在实际应用中日益普及，但仍易生成有害内容。从绕过安全过滤的对抗性越狱攻击，到难以检测的隐性仇恨言论，模型带来的风险持续扩大。尽管专用内容审核器和通用大模型都被用作安全层，但何种模型最适合应对何种有害内容仍不清楚。我们提出了迄今为止最全面的LLM安全能力评估，系统测试了53个模型在11个数据集上的表现，这些数据集被划分为四类不同场景。在仅提示和提示-响应两种设置下，研究揭示了关键盲点：在某一类别领先的前沿大模型，在其他类别上明显落后于小型专用替代方案；所有模型家族在真实对话安全方面均未取得突破。这些发现挑战了‘规模即安全’的假设，并为社区提供了有依据的模型选择框架。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are increasingly deployed in real-world applications, yet they remain vulnerable to generating harmful content. From adversarial jailbreaks that bypass safety filters to implicit hate that evades detection, the range of risks these models pose continues to grow. While both specialized content moderators and general-purpose LLMs are being used as safety layers, the question of which model is best suited for which type of harmful content remains unanswered. We present the most comprehensive evaluation of LLM safety capabilities to date, systematically testing \textbf{53} models across \textbf{11} datasets that we organize into four distinct categories. Our evaluation under both prompt-only and prompt-response settings uncovers critical blind spots: large frontier models that lead on one category fall significantly behind smaller, specialized alternatives on others, and real-world conversational safety remains largely unsolved across all model families. These findings challenge the assumption that scale alone ensures safety, and provide the community with a structured framework for informed model selection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。