提出大模型知识体系三大瓶颈及一种可落地的未来范式
Open Problems and a Hypothetical Path Forward in LLM Knowledge Paradigms
- 针对知识更新难、逆向泛化失效、内部知识冲突三大问题
- 提出基于上下文知识扩展的假设性新范式，可兼容现有技术
- 适合关注大模型知识架构演进的研究者参考
知识是大语言模型整体能力的核心。模型的知识范式决定了其知识的编码与利用方式，显著影响性能表现。尽管当前大模型在既有知识范式下持续发展，但框架内仍存在诸多制约模型潜力的问题。本文指出三大关键开放问题：（1）大模型知识更新困难；（2）反向知识泛化失败（即‘逆转诅咒’）；（3）内部知识间的冲突。回顾近期在解决这些问题上的进展，并探讨潜在通用解决方案。基于这些观察，提出一种基于上下文知识扩展的假设性范式，并规划出在现有技术条件下可行的实现路径。证据表明该方法具备缓解当前缺陷的潜力，代表了未来模型范式的愿景。本文旨在为研究人员提供大模型知识系统进展的简要概述，同时激发下一代模型架构的发展思路。
原文摘要 · Abstract (English)
Knowledge is fundamental to the overall capabilities of Large Language Models (LLMs). The knowledge paradigm of a model, which dictates how it encodes and utilizes knowledge, significantly affects its performance. Despite the continuous development of LLMs under existing knowledge paradigms, issues within these frameworks continue to constrain model potential. This blog post highlight three critical open problems limiting model capabilities: (1) challenges in knowledge updating for LLMs, (2) the failure of reverse knowledge generalization (the reversal curse), and (3) conflicts in internal knowledge. We review recent progress made in addressing these issues and discuss potential general solutions. Based on observations in these areas, we propose a hypothetical paradigm based on Contextual Knowledge Scaling, and further outline implementation pathways that remain feasible within contemporary techniques. Evidence suggests this approach holds potential to address current shortcomings, serving as our vision for future model paradigms. This blog post aims to provide researchers with a brief overview of progress in LLM knowledge systems, while provide inspiration for the development of next-generation model architectures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。