大模型中的概念在上下文中会动态变化，且不同模型共享这种变换几何结构。
Language Models Represent and Transform Concepts with Shared Geometry
- 将概念视为点云流形，上下文视为向量场来建模变换
- 不同概念受上下文影响的偏移量差异具有语义组织性
- 跨模型的变换结构可相互预测，表明共享几何存在
神经网络中概念如何表征是机器学习的基础问题。主流观点将概念表征视为静态几何对象，但概念总在上下文中出现，且上下文会改变它们。基于神经种群几何，我们形式化地将概念表征为点云流形，上下文变换为向量场，并在六种不同规模的大语言模型中验证。结果发现，上下文对每个概念的移动方式各不相同，这些位移的方差具有语义组织性，与词汇具体性和密度相关。更重要的是，被变换的概念及其方差结构在模型间共享：一个模型的位移结构能显著高于随机水平地预测另一模型的未见位移。这表明模型不仅在概念表征上共享几何结构，更在上下文变换机制上存在深层共性，其组织性远超以往认知。
原文摘要 · Abstract (English)
How concepts are represented in neural networks is a fundamental question in machine learning. The dominant view treats concept representations as stationary geometric objects. Yet concepts appear in context, and context transforms them. Drawing from neural population geometry, we formalize concept representations as point-cloud manifolds and contextual transformations as vector fields, and instantiate this framework in large language models. Across six model families of varying scales, we find that context moves each concept differently. The variance in these displacements is semantically organized, correlating with lexical concreteness and density. Importantly, both the concepts being transformed and this variance structure are shared across models: displacement structure transported from one model predicts held-out displacements in others significantly above chance. Together, these findings show that models share a common geometry not only in how concepts are represented, but more importantly in how context transforms them, a structure with richer organization than prior work has recognized.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。