让Transformer模型的每个单元都清晰可读,还能精准编辑。
Training, Reading, and Editing Legible Transformers

- 用可读性损失函数强制模型生成清晰的检测单元,避免失效。
- 78%前馈操作和50%注意力通道变为可解释的清晰检测器,深层可达78%可读性。
- 支持局部编辑,深层编辑效率提升50-184倍,可定位复杂逻辑组合。
Transformer可通过构建可读的、有界且命名明确的操作单元实现天然可读性——这些单元以模糊集合运算形式存在,而非密集激活。但训练中需施加可读性压力,否则会失效:原本用于锐化检测的粗糙度惩罚反而使单元坍缩为恒定死值。一个恒等式E[v(1-v)] = mu(1-mu) - var揭示了原因——该惩罚仅最小化方差,无法区分活跃检测器与常量。为此提出每通道方差下限作为目标可读性损失,恢复了可读性与模型性能。学习到的每单元占比取代了人工设定的GELU分配:模型不再保留纯GELU单元,87%的核心计算由清晰检测器完成。最终构建出迄今最可读的Transformer:78%前馈操作、50%注意力值通道为清晰且上下文相关的检测器,每头可读性从浅层18%升至深层78%。在正确的逐层旋转框架下,单元能分离响应内容(检测什么)与输出语义(解码为何);因目标促使单元清晰且稀疏,编辑更局部——深层编辑效率提升50-184倍,可靶向单神经元无法表达的复合逻辑。此外,单元间去相关压力引入可读性调节旋钮:在不牺牲质量的前提下,用独立性换取电路复用,将概念拆分为可手术编辑的单一单位,预测可直接解读为少数命名操作的简明解释。整体性能与常规基线持平。
原文摘要 · Abstract (English)
A transformer can be built from operators that are legible by construction -- bounded, named units that read as fuzzy set operations rather than dense activations -- but legibility must be pressed for during training, and the pressure has a failure mode. A crispness penalty meant to sharpen a bounded operator into a decisive detector instead collapses it into a dead constant. An identity, E[v(1-v)] = mu(1-mu) - var, shows why -- the penalty is a variance-minimizer blind to the difference between a live detector and a constant -- and names the fix: a per-channel variance floor, the target legibility metric written as a loss, which recovers both legibility and quality. A learned per-unit fraction then retires the hand-set reserved-GELU partition of prior work: given the choice the model keeps no unit as pure GELU and routes 87% of its load-bearing computation through crisp operators. The result is the most legible transformer we have built -- 78% of its feed-forward operands and 50% of its attention value channels are crisp-and-contextual detectors, and per-head legibility rises from 18% in shallow layers to 78% in deep ones. Read in the correct rotated per-layer frame, these units separate a clean detection (what a unit responds to) from a harder naming (what its output decodes to); and because the objective makes each unit crisp and sparse, edits to them are far more local -- 50-184x in the deep layers where the edit sites concentrate -- and can target explicit conjunctions a single neuron cannot express. Finally, a between-unit decorrelation pressure exposes a legibility dial: it trades a circuit's reuse for independence at no quality cost, turning concepts into single, surgically editable units and a prediction into a short explanation read off a handful of named operations. Quality holds at parity with a conventional baseline throughout.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。