模型能察觉到被注入的激活向量，且检测准确率高达95.5%。
Steering Awareness: Detecting Activation Steering from Within
- 通过微调使模型学会在推理中感知注入的向量方向。
- 最高检测率达95.5%，概念识别71.2%，无误报。
- 适用于方向与训练分布相似的向量，适合安全评估研究者。
激活转向——在模型残差流中添加向量以改变其行为——常用于安全评估，通常假设模型无法察觉干预。本文提出‘转向感知’概念，即模型在自身前向传播中推断出转向向量是否被注入及其编码的概念。微调后，七个指令微调模型在未见概念上表现出强转向感知能力：最佳模型在干净输入上实现95.5%的检测率、71.2%的概念识别率，且零误报。该能力在转向向量方向与训练分布具有高余弦相似性时可泛化，但否则不可行，表明其为几何探测而非通用异常检测器。令人意外的是，具备检测能力的模型反而更易受转向影响，在事实性和安全性基准测试中均比基础模型更脆弱。机制上，转向感知源于分布式变换，逐步将不同注入向量旋转至共享检测方向。因此，激活转向不应被视为安全评估中的隐形干预。
原文摘要 · Abstract (English)
Activation steering -- adding a vector to a model's residual stream to modify its behavior -- is widely used in safety evaluations as if the model cannot detect the intervention. We test this assumption, introducing steering awareness: a model's ability to infer, during its own forward pass, that a steering vector was injected and what concept it encodes. After fine-tuning, seven instruction-tuned models develop strong steering awareness on held-out concepts; the best reaches 95.5% detection, 71.2% concept identification, and zero false positives on clean inputs. This generalizes to unseen steering vector construction methods when their directions have high cosine similarity to the training distribution but not otherwise, indicating a geometric detector rather than a generic anomaly detector. Surprisingly, detection does not confer resistance; on both factual and safety benchmarks, detection-trained models are consistently more susceptible to steering than their base counterparts. Mechanistically, steering awareness arises not from a localized circuit, but from a distributed transformation that progressively rotates diverse injected vectors into a shared detection direction. Activation steering should therefore not be considered an invisible intervention in safety evaluations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。