语言模型通过分层剪裁生成新概念，揭示人工认知的形成机制。
The Process of Categorical Clipping at the Core of the Genesis of Concepts in Synthetic Neural Cognition
- 分层神经网络通过三因素剪裁旧类别生成新类别
- 新类别由前层类别提取特定子维度构成，形成形式与背景区分
- 适用于研究大模型内部概念演化过程
本文在人工智能神经心理学领域，研究语言模型执行的类别分割过程。该过程在不同神经层中生成新的功能类别维度，以分析输入文本并完成任务。多层感知机（MLP）中的每个神经元对应一个特定类别，由神经聚合函数携带的三因素决定：类别引导、类别注意力和类别相位。每一新层均基于前层神经元的类别，通过类别剪裁形成新类别——即从先前类别中选择性提取特定子维度，构建形式与类别背景之间的差异。我们探索了这一合成剪裁的多个认知特性：类别压缩、类别选择性、初始嵌入维度分离及类别区域分割。
原文摘要 · Abstract (English)
This article investigates, within the field of neuropsychology of artificial intelligence, the process of categorical segmentation performed by language models. This process involves, across different neural layers, the creation of new functional categorical dimensions to analyze the input textual data and perform the required tasks. Each neuron in a multilayer perceptron (MLP) network is associated with a specific category, generated by three factors carried by the neural aggregation function: categorical priming, categorical attention, and categorical phasing. At each new layer, these factors govern the formation of new categories derived from the categories of precursor neurons. Through a process of categorical clipping, these new categories are created by selectively extracting specific subdimensions from the preceding categories, constructing a distinction between a form and a categorical background. We explore several cognitive characteristics of this synthetic clipping in an exploratory manner: categorical reduction, categorical selectivity, separation of initial embedding dimensions, and segmentation of categorical zones.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。