arXiv:2602.00959cs.LGcs.CL2026-02

用交互式智能体系统挖掘大模型真实知识边界,发现模型越大越能提取更多知识。

Probing the Knowledge Boundary: An Interactive Agentic Framework for Deep Knowledge Extraction

  • 设计四种自适应探索策略,分层探测模型知识
  • 大模型知识量随规模增长,存在明确扩展规律
  • 适合研究模型知识构成与训练数据影响的学者

大语言模型可视为压缩的知识库,但其实际包含的知识内容及知识边界的范围仍不清晰。现有基准多为静态,难以支持系统的知识探测。本文提出一种交互式智能体框架,用于系统性地提取与量化大模型的知识。方法包含四种自适应探索策略,以不同粒度探测知识。为保证提取质量,引入三阶段知识处理流程:基于向量的去重、利用LLM解决语义模糊重叠、领域相关性审计保留有效知识单元。大量实验表明,递归分类法是最有效的探索策略;观察到明显的知识扩展规律,更大模型持续恢复更多知识。此外,发现Pass@1与Pass@k之间存在权衡:领域专用模型初始准确率高但快速退化,通用模型则性能稳定。最终结果表明,训练数据组成差异导致不同模型家族具有显著且可测量的知识特征,反映出预训练如何塑造模型的参数化知识。

原文摘要 · Abstract (English)

Large Language Models (LLMs) can be seen as compressed knowledge bases, but it remains unclear what knowledge they truly contain and how far their knowledge boundary extends. Existing benchmarks are mostly static and provide limited support for systematic knowledge probing. In this paper, we propose an interactive agentic framework to systematically extract and quantify the knowledge of LLMs. Our method includes four adaptive exploration policies to probe knowledge at different granularity. To ensure the quality of extracted knowledge, we introduce a three-stage knowledge processing pipeline that combines vector-based filtering to remove strict duplicates, LLM-based adjudication to resolve ambiguous semantic overlap, and domain relevance auditing to retain valid knowledge units. Through extensive experiments, we find that Recursive Taxonomy is the most effective exploration strategy. We also observe a clear knowledge scaling law, where larger models consistently recover more knowledge. In addition, we identify a Pass@1 versus Pass@k trade-off: domain-specialized models achieve higher initial accuracy but experience rapid degradation, while general-purpose models maintain stable performance over extended extraction. Finally, our results show that differences in training data composition lead to distinct and measurable knowledge profiles across model families, reflecting how pretraining shapes each model's parametric knowledge.

知识挖掘大模型智能体

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。