arXiv:2501.00226cs.AIcs.CL2025-01被引 23

大模型通过社会性语言编码,间接学习世界模型。

Generative Emergent Communication: Large Language Model is a Collective World Model

  • 将语言生成视为多智能体的分布式贝叶斯推断过程。
  • 模型通过解码人类语言重建集体认知表征,形成隐空间结构。
  • 为大模型能力提供数学解释,适合理解认知与语言演化者。

大型语言模型(LLM)虽未直接具备感官运动经验,却展现出惊人的世界知识获取能力,其机制仍属根本谜题。本文提出‘集体世界模型’假说:LLM并非从零构建世界模型,而是学习人类语言中已由社会长期交互意义建构所隐含的统计近似。为此,我们引入生成式涌现通信(Generative EmCom)框架,基于集体预测编码(CPC)建模语言作为多智能体去中心化贝叶斯推断的产物。该过程在社会尺度上形成编码-解码结构:人类社会将具身认知表征编码至语言,而LLM则解码这些符号以重构原始集体表征的潜在空间。此视角为大模型能力提供了原则性、数学化的解释。主要贡献包括:1)形式化Generative EmCom框架,阐明其与世界模型及多智能体强化学习的关系;2)用于解释大模型现象,如分布语义是表征重建的自然结果。本工作统一了个体认知发展、集体语言演化与大规模人工智能基础。

原文摘要 · Abstract (English)

Large Language Models (LLMs) have demonstrated a remarkable ability to capture extensive world knowledge, yet how this is achieved without direct sensorimotor experience remains a fundamental puzzle. This study proposes a novel theoretical solution by introducing the Collective World Model hypothesis. We argue that an LLM does not learn a world model from scratch; instead, it learns a statistical approximation of a collective world model that is already implicitly encoded in human language through a society-wide process of embodied, interactive sense-making. To formalize this process, we introduce generative emergent communication (Generative EmCom), a framework built on the Collective Predictive Coding (CPC). This framework models the emergence of language as a process of decentralized Bayesian inference over the internal states of multiple agents. We argue that this process effectively creates an encoder-decoder structure at a societal scale: human society collectively encodes its grounded, internal representations into language, and an LLM subsequently decodes these symbols to reconstruct a latent space that mirrors the structure of the original collective representations. This perspective provides a principled, mathematical explanation for how LLMs acquire their capabilities. The main contributions of this paper are: 1) the formalization of the Generative EmCom framework, clarifying its connection to world models and multi-agent reinforcement learning, and 2) its application to interpret LLMs, explaining phenomena such as distributional semantics as a natural consequence of representation reconstruction. This work provides a unified theory that bridges individual cognitive development, collective language evolution, and the foundations of large-scale AI.

大模型语言演化认知科学集体智能

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。