同一模型在不同随机种子下对有害请求的拒绝行为不一致，影响安全评估可靠性。
The Instability of Safety: How Random Seeds and Temperature Expose Inconsistent LLM Refusal Behavior
- 通过多组随机种子和温度测试，发现模型对相同有害提示的拒绝行为会随设置变化
- 18%-28%的提示出现拒绝/合规翻转，高温显著降低决策稳定性
- 建议采用多样本评估，避免单次测试误判，尤其在高温度时需增加采样数
当前大语言模型的安全评估依赖单次测试，隐含假设模型响应是确定且具代表性的。我们挑战这一假设，研究了在不同随机种子和温度设置下模型拒绝行为的稳定性。在三个模型家族的四款指令微调模型（Llama 3.1 8B、Qwen 2.5 7B、Qwen 3 8B、Gemma 3 12B）上，对876个有害提示在20种采样配置（4种温度×5个种子）下测试，发现18%-28%的提示出现决策翻转——即某些配置下拒绝，另一些则合规。我们的安全稳定性指数（SSI）显示，高温显著降低稳定性（Friedman卡方=396.81，p<0.001），平均内部温度SSI从温度0.0的0.977降至1.0的0.942。使用Claude 3.5 Haiku作为统一外部评判者验证结果，与Llama 70B评判者达成89.1%的一致性（Cohen's kappa=0.62）。各模型内部，合规率更高的提示稳定性更低（Spearman rho=-0.47至-0.70，均p<0.001），表明模型在边界请求上更易动摇。结果表明，单次测试不足以可靠评估安全性，评估协议必须考虑模型行为的随机波动。以Llama 3.1 8B为例，单次测试与多样本真实情况一致率仅92.5%（贪心为98.7%，温度1.0时降至90.3%），我们建议按温度调整采样量——贪心时1次，低温3次，高温更多（3次仅达~95%），合并时建议10次以上。
原文摘要 · Abstract (English)
Current safety evaluations of large language models rely on single-shot testing, implicitly assuming that model responses are deterministic and representative of the model's safety alignment. We challenge this assumption by investigating the stability of safety refusal decisions across random seeds and temperature settings. Testing four instruction-tuned models from three families (Llama 3.1 8B, Qwen 2.5 7B, Qwen 3 8B, Gemma 3 12B) on 876 harmful prompts across 20 sampling configurations (4 temperatures x 5 seeds), we find that 18-28% of prompts exhibit decision flips--the model refuses in some configurations but complies in others--depending on the model. Our Safety Stability Index (SSI) reveals that higher temperatures significantly reduce decision stability (Friedman chi-squared = 396.81, p < 0.001), with mean within-temperature SSI dropping from 0.977 at temperature 0.0 to 0.942 at temperature 1.0. We validate findings across all model families using Claude 3.5 Haiku as a unified external judge, achieving 89.1% inter-judge agreement with the Llama 70B judge on the two models both judges labeled (Cohen's kappa = 0.62). Within each model, prompts with higher compliance rates exhibit lower stability (Spearman rho = -0.47 to -0.70, all p < 0.001), indicating that models "waver" more on borderline requests. These findings demonstrate that single-shot safety evaluations are insufficient for reliable safety assessment and that evaluation protocols must account for stochastic variation in model behavior. For Llama 3.1 8B, single-shot evaluation agrees with multi-sample ground truth only 92.5% of the time when pooling across temperatures (98.7% at greedy to 90.3% at temperature 1.0), and we recommend scaling samples with temperature--one at greedy, three at low temperature, more at higher temperatures (where three reach only ~95%), and ten when pooling--rather than a single flat threshold.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。