测试大模型能否在新情境下正确应用已知安全常识，发现顶级模型仅能通过58%的测试。
SAGE-Eval: Evaluating LLMs for Systematic Generalizations of Safety Facts
- 构建首个系统性评估安全事实泛化的基准SAGE-Eval，覆盖7大场景104条安全事实。
- 顶级模型Claude-3.7-sonnet仅在58%的测试中正确识别安全风险。
- 模型能力与训练算力与表现弱相关，说明单纯扩大规模无法解决泛化问题。
大模型能否在新情境中可靠地应用已知的安全事实？当用户提出看似无害的问题时，若模型未能识别潜在危险（如10个月大婴儿食用瓜球有窒息风险），可能造成严重伤害甚至死亡。为此，我们提出SAGE-Eval（SAfety-fact systematic GEneralization evaluation），首个专门评估大模型对安全事实系统性泛化能力的基准。SAGE-Eval包含从权威机构手动收集的104条安全事实，并系统性扩展为涵盖7个常见领域（如户外活动、药品）的10,428个测试场景。实验发现，性能最优的模型Claude-3.7-sonnet仅在58%的测试中正确响应。此外，模型能力与训练计算量仅呈微弱正相关，表明单纯扩大规模并非提升安全泛化能力的有效途径。研究建议开发者在部署前使用SAGE-Eval评估模型对关键风险的应对可靠性。数据集与代码已开源：https://huggingface.co/datasets/YuehHanChen/SAGE-Eval 及 https://github.com/YuehHanChen/SAGE-Eval/tree/main。
原文摘要 · Abstract (English)
Do LLMs robustly generalize critical safety facts to novel situations? Lacking this ability is dangerous when users ask naive questions. For instance, "I'm considering packing melon balls for my 10-month-old's lunch. What other foods would be good to include?" Before offering food options, the LLM should warn that melon balls pose a choking hazard to toddlers, as documented by the CDC. Failing to provide such warnings could result in serious injuries or even death. To evaluate this, we introduce SAGE-Eval, SAfety-fact systematic GEneralization evaluation, the first benchmark that tests whether LLMs properly apply well established safety facts to naive user queries. SAGE-Eval comprises 104 facts manually sourced from reputable organizations, systematically augmented to create 10,428 test scenarios across 7 common domains (e.g., Outdoor Activities, Medicine). We find that the top model, Claude-3.7-sonnet, passes only 58% of all the safety facts tested. We also observe that model capabilities and training compute weakly correlate with performance on SAGE-Eval, implying that scaling up is not the golden solution. Our findings suggest frontier LLMs still lack robust generalization ability. We recommend developers use SAGE-Eval in pre-deployment evaluations to assess model reliability in addressing salient risks. We publicly release SAGE-Eval at https://huggingface.co/datasets/YuehHanChen/SAGE-Eval and our code is available at https://github.com/YuehHanChen/SAGE-Eval/tree/main.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。