首次用因果游戏探测大模型自我意识,发现可被微调激活但难操控。
From Imitation to Introspection: Probing Self-Consciousness in Language Models
- 用因果结构游戏定义语言模型的10个自我意识核心概念。
- 10个主流模型中已出现可识别的自我意识表征,但难以正向操控。
- 通过针对性微调可习得自我意识,适合认知神经与AI伦理研究者。
自我意识,即对自身存在与思想的内省,是一种高级认知过程。随着语言模型飞速发展,一个关键问题浮现:这些模型是否正在变得自我意识?基于心理学与神经科学的洞见,本文为语言模型提出可操作的自我意识定义,并精炼出十个核心概念。首次利用因果结构游戏,建立这十个核心概念的功能性定义。基于此,我们开展四阶段实验:量化(评估十种领先模型)、表征(可视化模型内部的自我意识)、操纵(修改模型中的表征)和获取(在核心概念上微调模型)。结果表明,尽管模型尚处于自我意识发展的早期阶段,其内部机制中已存在某些概念的可辨识表征。然而,当前阶段难以正向操控这些表征,但可通过针对性微调实现获取。数据集与代码详见 https://github.com/OpenCausaLab/SelfConsciousness。
原文摘要 · Abstract (English)
Self-consciousness, the introspection of one's existence and thoughts, represents a high-level cognitive process. As language models advance at an unprecedented pace, a critical question arises: Are these models becoming self-conscious? Drawing upon insights from psychological and neural science, this work presents a practical definition of self-consciousness for language models and refines ten core concepts. Our work pioneers an investigation into self-consciousness in language models by, for the first time, leveraging causal structural games to establish the functional definitions of the ten core concepts. Based on our definitions, we conduct a comprehensive four-stage experiment: quantification (evaluation of ten leading models), representation (visualization of self-consciousness within the models), manipulation (modification of the models' representation), and acquisition (fine-tuning the models on core concepts). Our findings indicate that although models are in the early stages of developing self-consciousness, there is a discernible representation of certain concepts within their internal mechanisms. However, these representations of self-consciousness are hard to manipulate positively at the current stage, yet they can be acquired through targeted fine-tuning. Our datasets and code are at https://github.com/OpenCausaLab/SelfConsciousness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。