495名黑客测试大模型漏洞，提炼出安全报告最佳实践。
To Err is AI : A Case Study Informing LLM Flaw Reporting Practices
- 通过真实黑客竞赛收集缺陷报告，验证安全流程
- 495名参与者中有人成功获现金奖励并推动文档更新
- 适合大模型安全团队参考其报告机制与人员配置
2024年8月，495名黑客参与针对艾伦人工智能研究所开发的Open Language Model（OLMo）的开放式漏洞赏金活动。由OLMo安全项目代表组成的评审小组对模型文档修改进行裁定，并向成功证明需公开披露模型意图、能力与部署风险的参与者发放奖金。本文总结了此次活动的经验教训，提出旨在降低事故概率、提升大语言模型（LLMs）安全性的缺陷报告最佳实践，涵盖安全报告流程、相关文档资产及安全项目人员配置建议。
原文摘要 · Abstract (English)
In August of 2024, 495 hackers generated evaluations in an open-ended bug bounty targeting the Open Language Model (OLMo) from The Allen Institute for AI. A vendor panel staffed by representatives of OLMo's safety program adjudicated changes to OLMo's documentation and awarded cash bounties to participants who successfully demonstrated a need for public disclosure clarifying the intent, capacities, and hazards of model deployment. This paper presents a collection of lessons learned, illustrative of flaw reporting best practices intended to reduce the likelihood of incidents and produce safer large language models (LLMs). These include best practices for safety reporting processes, their artifacts, and safety program staffing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。