AI通过逐层重构语义类别，逐步学会更高效地理解语言世界。
Synthetic Categorical Restructuring large Or How AIs Gradually Extract Efficient Regularities from Their Experience of the World
- 逐层抽象整合前层语义子维度，生成更高效的分类体系。
- 可视化显示GPT2-XL从第0层到第1层的类别重构过程。
- 揭示大模型如何从经验中提取规律，适合研究AI认知机制者阅读。
语言模型如何对其内部的语言世界经验进行分割，以逐步提高交互效率？本研究从人工智能神经心理学角度，探讨了合成类别重构现象：每一层感知机神经元均从上一层的思想类别中抽象并组合相关语义子维度，从而构建出更高效的新类别，用于分析和处理自身所接触的语义外部世界。本研究配套的基因神经元观察器，可可视化GPT2-XL中从第0层到第1层的合成类别重构过程。
原文摘要 · Abstract (English)
How do language models segment their internal experience of the world of words to progressively learn to interact with it more efficiently? This study in the neuropsychology of artificial intelligence investigates the phenomenon of synthetic categorical restructuring, a process through which each successive perceptron neural layer abstracts and combines relevant categorical sub-dimensions from the thought categories of its previous layer. This process shapes new, even more efficient categories for analyzing and processing the synthetic system's own experience of the linguistic external world to which it is exposed. Our genetic neuron viewer, associated with this study, allows visualization of the synthetic categorical restructuring phenomenon occurring during the transition from perceptron layer 0 to 1 in GPT2-XL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。