通过梯度子空间动态检测大模型知识缺口，提升问答可信度。
GRADE: Probing Knowledge Gaps in LLMs through Gradient Subspace Dynamics
- 用梯度与隐藏状态子空间的秩比量化知识激活程度。
- 在6个基准上验证有效，对输入扰动保持鲁棒性。
- 可解释长文本回答中的知识缺失，适合模型可解释性研究者。
检测大模型内部知识是否足以正确回答问题是部署负责任大模型的核心挑战。除依赖模型自报告置信度外，近期方法尝试通过响应标记的隐藏状态捕捉激活的知识量。我们指出，此类激活的知识可能与问题需求不匹配，例如包含风格和长度等无关特征。为此，提出GRADE（GRAdient Dynamics for knowledge gap detection），通过梯度与对应隐藏状态子空间的跨层秩比量化知识缺口。该方法基于梯度作为目标知识更新估计器的性质。在六个基准上验证了GRADE的有效性与对输入扰动的鲁棒性。此外，案例研究展示梯度链可生成长文本回答中知识缺口的可解释说明。代码已公开于 https://github.com/yjEugenia/llm-aware。
原文摘要 · Abstract (English)
Detecting whether a model's internal knowledge is sufficient to correctly answer a given question is a fundamental challenge in deploying responsible LLMs. In addition to verbalising the confidence by LLM self-report, more recent methods explore the model internals, such as the hidden states of the response tokens, to capture how much knowledge is activated. We argue that such activated knowledge may not align with what the query requires, e.g., capturing the stylistic and length-related features that are uninformative for answering the query. To fill the gap, we propose GRADE (GRAdient Dynamics for knowlEdge gap detection), which quantifies the knowledge gap via the cross-layer rank ratio of the gradient to that of the corresponding hidden state subspace. This is motivated by the property of gradients as estimators of the required knowledge updates for a given target. We validate GRADE on six benchmarks, demonstrating its effectiveness and robustness to input perturbations. In addition, we present a case study demonstrating how the gradient chain can generate interpretable explanations of knowledge gaps in long-form answers. Code is available at https://github.com/yjEugenia/llm-aware
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。