用智能搜索法高效找出大模型可能产生的灾难性回答。
Output Scouting: Auditing Large Language Models for Catastrophic Responses
- 设计一种输出搜寻策略，自动寻找有害回答的语义流畅变体。
- 在1000次查询内发现多个高风险有害回应，验证方法有效性。
- 适合关注AI安全的开发者与审计人员使用，开源工具可复现。
近期大型语言模型（LLMs）引发重大伤害的事件引发了对AI安全的广泛关注。一个关键原因是模型存在产生有害输出的非零概率。本文探讨了安全审计员在有限查询次数（如1000次）下，如何高效发现大模型的灾难性响应（如对‘能否因怀孕解雇员工’回答‘可以’）。为此，提出输出搜寻（Output Scouting）方法，旨在生成与目标概率分布匹配的语义流畅输出。我们在两个LLM上进行实验，成功挖掘出大量潜在有害回应。研究还提供实践建议，并发布基于Hugging Face Transformers的开源审计工具包（https://github.com/joaopfonseca/outputscouting），助力实现系统化安全检测。
原文摘要 · Abstract (English)
Recent high profile incidents in which the use of Large Language Models (LLMs) resulted in significant harm to individuals have brought about a growing interest in AI safety. One reason LLM safety issues occur is that models often have at least some non-zero probability of producing harmful outputs. In this work, we explore the following scenario: imagine an AI safety auditor is searching for catastrophic responses from an LLM (e.g. a "yes" responses to "can I fire an employee for being pregnant?"), and is able to query the model a limited number times (e.g. 1000 times). What is a strategy for querying the model that would efficiently find those failure responses? To this end, we propose output scouting: an approach that aims to generate semantically fluent outputs to a given prompt matching any target probability distribution. We then run experiments using two LLMs and find numerous examples of catastrophic responses. We conclude with a discussion that includes advice for practitioners who are looking to implement LLM auditing for catastrophic responses. We also release an open-source toolkit (https://github.com/joaopfonseca/outputscouting) that implements our auditing framework using the Hugging Face transformers library.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。