揭示了Transformer中概念在深层逐步形成的过程，提出概念分配区新框架。
The Concept Allocation Zone: Tracking How Concepts Form Across Transformer Depth
- 定义概念分配区（CAZ），用三类指标量化概念在深度中的可分性变化。
- 发现概念分离曲线常呈多峰态，存在传统方法无法识别的微弱分配区。
- 适合研究模型内部表示演化、可解释性与深度特征组织的学者使用。
Transformer语言模型中的概念形成是一个沿深度扩展的过程，而非单层事件：一个概念在残差流中一个或多个连续区域变得可分，这一区域称为概念分配区（CAZ）。CAZ并非概念本身，而是模型在该深度段组织几何结构以使概念可分的区域——多个概念可能共享一个CAZ，且通常跨越多个深度段；配套的GEM论文表明，分离方向在CAZ内持续旋转，直至超过边界才趋于稳定。本文通过三种逐层指标——分离度、概念一致性与概念速度——形式化定义了CAZ，采用无显著性阈值的自动化边界检测（每个片段均为CAZ，“强”与“弱”仅由得分区分，非二元切割）。在35个模型、8种架构族和7个概念上的实证验证显示，分离度曲线S(l)常为多模态，得分检测揭示出传统峰值检测无法捕捉的细微分配区域（“微弱CAZ”）。该框架生成七项可检验预测，其贡献在于所提出的工具与揭示的现象——得分检测器、三项指标及多峰/微弱CAZ发现，而非预测本身。代码已开源发布为 rosetta_tools 库（v1.3.1）。
原文摘要 · Abstract (English)
Concept formation in transformer language models is a depth-extended process, not a single-layer event: a concept becomes separable across one or more contiguous regions of the residual stream - its Concept Allocation Zone (CAZ). A CAZ is not a concept but the depth segment where the model organizes its geometry to make one separable - concepts may share a CAZ, and typically span multiple across depth; the companion GEM paper shows the separating direction continues to rotate within a CAZ before stabilizing past its boundary. We formalize the CAZ through three layer-wise metrics - Separation, Concept Coherence, and Concept Velocity - with automated boundary detection that applies no significance threshold to CAZ membership (every segment is a CAZ; "strong" vs. "gentle" is score, never a binary cut). Empirical validation across 35 models, 8 architectural families, and 7 concepts shows the separation curve S(l) is frequently multimodal, and scored detection surfaces a further category of subtle allocation regions ("gentle CAZes") invisible to standard peak detection. The framework generates seven testable predictions; its contribution is the instrument and the phenomena it surfaces - the scored detector, the three metrics, and the multimodal/gentle-CAZ findings - not the predictions themselves. Released as the open-source rosetta_tools library (v1.3.1).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。