LLM不是世界模型的对立面,而是其退化特例,存在连续进化路径。
From Tokens to States: LLMs as a Special Case of World Models and the Continuous Path Beyond
- 将语言模型视为状态空间中的特殊世界模型,用连续路径替代非此即彼的二元对立
- 从单标记预测到未来摘要预测,再到下一潜在状态预测,逐步放松语言模型约束
- 揭示训练可扩展性的两大支柱(海量文本数据与变压器架构)如何随演进而消解
AI社区常将大语言模型(LLMs)与世界模型对立:前者预测离散标记,后者模拟现实。2022年杨·勒昆提出,实现通用智能需放弃自回归标记预测,转向潜在空间架构。这一观点过于二元。本文主张两点:第一,LLMs是世界模型的一种退化特例——状态空间为所有标记序列,唯一动作是添加一个标记,因此世界模型是对LLMs的严格泛化而非替代;第二,从单标记预测(NTP)到联合嵌入预测架构(JEPA)之间存在自然连续谱系,包含多标记预测、未来摘要预测和下一潜在状态预测等中间阶段,这些已在现有研究中出现。沿此路径推进可逐项放宽LLM的约束,同时逐步失去当前使LLM可大规模训练的两大优势:互联网规模的自监督文本数据,以及专为离散标记预测设计的Transformer架构。这两方面均被列为开放研究问题:数据问题(自监督文本到带标签动作环境的断崖式跨越),架构问题(Transformer能否泛化至连续状态预测,或需新基础组件)。
原文摘要 · Abstract (English)
The AI community has framed the relationship between large language models (LLMs) and world models as a dichotomy: LLMs predict tokens; world models simulate reality. Yann LeCun argues in 2022 that reaching general intelligence requires abandoning autoregressive token prediction in favour of latent-space architectures. This framing is unnecessarily binary. Two claims will be defended. First, LLMs are a degenerate special case of world models: the state space is the set of all token sequences, the only action is appending one token, and world models are therefore a strict generalisation of LLMs, not a replacement. Second, there is a natural continuous spectrum from NTP to JEPA, with multi-token prediction, future-summary prediction, and next-latent prediction as intermediate stations already populated by current research. Moving along this spectrum relaxes the LLM constraints one by one. It also progressively surrenders the two practical advantages that make LLMs trainable at scale: internet-scale self-supervised data, and a transformer architecture co-designed for discrete token prediction. Both are examined as open research questions: the data question (the cliff from self-supervised text to instrumented action-labelled environments) and the architecture question (whether the transformer generalises to continuous-state prediction, or whether a new primitive is needed).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。