用强化学习自动探测大模型的知识边界，发现它能答和不能答的问题。
Knowledge Boundary Discovery for Large Language Models
- 设计强化学习代理，通过提问逐步探索模型回答能力的边界。
- 生成的可答与不可答问题集与人工构建数据集相当，验证有效性。
- 适合评估大模型知识范围，尤其在幻觉检测和模型可靠性研究中实用。
我们提出知识边界发现（KBD），一种基于强化学习的框架，用于探索大语言模型（LLM）的知识边界。通过自动生成两类问题：一类是模型能自信回答的（在边界内），另一类是其无法回答的（在边界外）。由于幻觉现象的存在，迭代探索和利用模型响应以定位知识边界极具挑战性。在部分可观测环境建模下，智能体作为行动生成渐进式问题，采用熵减少作为奖励，接收模型响应作为观测并更新信念状态。实验表明，KBD能够自动识别一组非平凡的可答与不可答问题，有效揭示模型知识边界。通过与人工构建的基准数据集对比验证，所生成的问题集与人类标注数据具有可比性。该方法为大模型评估提供了新路径。
原文摘要 · Abstract (English)
We propose Knowledge Boundary Discovery (KBD), a reinforcement learning based framework to explore the knowledge boundaries of the Large Language Models (LLMs). We define the knowledge boundary by automatically generating two types of questions: (i) those the LLM can confidently answer (within-knowledge boundary) and (ii) those it cannot (beyond-knowledge boundary). Iteratively exploring and exploiting the LLM's responses to find its knowledge boundaries is challenging because of the hallucination phenomenon. To find the knowledge boundaries of an LLM, the agent interacts with the LLM under the modeling of exploring a partially observable environment. The agent generates a progressive question as the action, adopts an entropy reduction as the reward, receives the LLM's response as the observation and updates its belief states. We demonstrate that the KBD detects knowledge boundaries of LLMs by automatically finding a set of non-trivial answerable and unanswerable questions. We validate the KBD by comparing its generated knowledge boundaries with manually crafted LLM benchmark datasets. Experiments show that our KBD-generated question set is comparable to the human-generated datasets. Our approach paves a new way to evaluate LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。