通过多智能体强化学习，仅用少量查询就能识别大模型在哪些话题上不可信。
Can We Trust a Black-box LLM? LLM Untrustworthy Boundary Detection via Bias-Diffusion and Multi-Agent Reinforcement Learning
- 用知识图谱结合多智能体强化学习，自动探测模型不可信话题边界。
- 仅需少量提问即可定位大模型产生偏见的敏感话题节点。
- 适合关注模型安全与可信度的研究者或应用开发者使用。
大型语言模型在众多领域展现出强大的问答能力，但有时会生成带有偏见、意识形态化或错误的回答，若无法明确判断其可信边界，则限制了实际应用。本文提出一种新算法GMRL-BD，可在仅黑箱访问模型且受查询约束的条件下，识别给定模型在特定话题上的不可信边界。基于从维基百科构建的通用知识图谱（KG），算法利用多个强化学习智能体协同探索，高效定位可能导致模型产生偏见的答案的话题节点。实验表明，该方法仅需有限次数的查询即可准确检测出不可信边界。此外，研究还发布了包含Llama2、Vicuna、Falcon、Qwen2、Gemma2和Yi-1.5等主流模型的新数据集，标注了各模型易产生偏见的具体话题。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have shown a high capability in answering questions on a diverse range of topics. However, these models sometimes produce biased, ideologized or incorrect responses, limiting their applications if there is no clear understanding of which topics their answers can be trusted. In this research, we introduce a novel algorithm, named as GMRL-BD, designed to identify the untrustworthy boundaries (in terms of topics) of a given LLM, with black-box access to the LLM and under specific query constraints. Based on a general Knowledge Graph (KG) derived from Wikipedia, our algorithm incorporates with multiple reinforcement learning agents to efficiently identify topics (some nodes in KG) where the LLM is likely to generate biased answers. Our experiments demonstrated the efficiency of our algorithm, which can detect the untrustworthy boundary with just limited queries to the LLM. Additionally, we have released a new dataset containing popular LLMs including Llama2, Vicuna, Falcon, Qwen2, Gemma2 and Yi-1.5, along with labels indicating the topics on which each LLM is likely to be biased.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。