整合评估与诊断的开源安全工具,让大模型风险可查可改
DeepSight: An All-in-One LM Safety Toolkit
- 构建评估-诊断一体化框架,打通安全流程断点
- 支持前沿风险评测,实现从行为到机制的全链路分析
- 首个开源联合评估诊断工具,适合安全研究与工程落地
随着大模型(LMs)快速发展,其安全性日益成为关键。当前大语言模型(LLMs)和多模态大语言模型(MLLMs)的安全工作流中,评估、诊断与对齐常由独立工具完成。现有安全评估仅能定位外部行为风险,无法揭示内部根因;而安全诊断则常脱离具体风险场景,停留在可解释层面。这导致对齐过程缺乏对内部机制变化的精准说明,可能损害模型通用能力。为此,我们提出开源项目 DeepSight,实践一种新的评估-诊断融合范式。DeepSight 由评估工具 DeepSafe 与诊断工具 DeepScan 构成,具备低成本、可复现、高效、高可扩展的特点。通过统一任务与数据协议,建立两阶段间的连接,将安全评估从黑箱转变为白箱洞察。此外,DeepSight 是首个支持前沿人工智能风险评估及联合安全评估与诊断的开源工具。
原文摘要 · Abstract (English)
As the development of Large Models (LMs) progresses rapidly, their safety is also a priority. In current Large Language Models (LLMs) and Multimodal Large Language Models (MLLMs) safety workflow, evaluation, diagnosis, and alignment are often handled by separate tools. Specifically, safety evaluation can only locate external behavioral risks but cannot figure out internal root causes. Meanwhile, safety diagnosis often drifts from concrete risk scenarios and remains at the explainable level. In this way, safety alignment lack dedicated explanations of changes in internal mechanisms, potentially degrading general capabilities. To systematically address these issues, we propose an open-source project, namely DeepSight, to practice a new safety evaluation-diagnosis integrated paradigm. DeepSight is low-cost, reproducible, efficient, and highly scalable large-scale model safety evaluation project consisting of a evaluation toolkit DeepSafe and a diagnosis toolkit DeepScan. By unifying task and data protocols, we build a connection between the two stages and transform safety evaluation from black-box to white-box insight. Besides, DeepSight is the first open source toolkit that support the frontier AI risk evaluation and joint safety evaluation and diagnosis.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。