通过分析模型内部表示，用拒绝对齐度量评估大模型安全性能。
RAS: Measuring LLM Safety Through Refusal Alignment
- 从安全对齐模型中提取分层拒绝方向，捕捉内部表征中的安全信号。
- 在多个模型族上验证，RAS得分与攻击成功率高度相关，且比人工评估快得多。
- 适合需要快速、可复现的白盒安全评估的研究者或工程团队使用。
大型语言模型（LLMs）的安全性评估通常通过输入不安全或越狱提示并判断输出是否违反安全策略来实现。尽管有效，这种基于输出的评估方式成本高、依赖评判者选择，且易受固定问题库限制。本文提出SafeVec，一种白盒评估方法，从内部表征而非生成结果中测量安全性。SafeVec首先从一个安全对齐的参考模型中提取分层拒绝方向，然后选取能分离安全与非安全行为的稳定层窗口，最后通过测量目标模型在非安全和越狱提示下隐藏状态与这些拒绝方向的一致性来评分。由此得到的指标RAS（拒绝对齐得分）将表示层面的拒绝对齐映射为0-100的校准安全分数。在Llama、Gemma和Qwen模型家族上，RAS能有效区分对齐与未过滤、消融的模型，追踪输出级攻击成功率，并显著快于基于人工评判的评估。结果表明，拒绝对齐是白盒评估大模型安全性的紧凑且高效信号。
原文摘要 · Abstract (English)
Safety evaluation of large language models (LLMs) is commonly performed by querying models with unsafe or jailbreak prompts and judging whether their outputs violate a safety policy. Although useful, output-level evaluation is expensive, sensitive to judge choice, and easily tied to fixed question banks. We propose **SafeVec**, a white-box evaluation procedure that measures safety from internal representations rather than generated answers. **SafeVec** first extracts layer-wise refusal directions from a safety-aligned reference model, then selects stable layer windows where safe and unsafe behaviors are separable, and finally scores a target model by measuring whether its hidden states align with these refusal directions under unsafe and jailbreak prompts. The resulting metric, **RAS** (**R**efusal **A**lignment **S**core), maps representation-level refusal alignment to a calibrated 0-100 safety score. Across `Llama`, `Gemma`, and `Qwen` model families, RAS separates aligned models from uncensored and abliterated variants, tracks output-level attack success rate, and is substantially faster than judge-based evaluation. These results suggest that refusal alignment provides a compact and efficient signal for white-box LLM safety evaluation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。