通过分析提示的内在维度，发现其能有效预测模型生成不确定性并识别恶意提示。
The Intrinsic Dimension of Prompts in Internal Representations of Large Language Models
- 用内在维度衡量提示在模型层间的几何结构变化。
- 内在维度在早期到中期层达到峰值，且与下一个词的不确定性高度相关。
- 基于该维度的探测器可提前识别恶意提示，准确率达90%~95%，优于现有防护工具。
我们从内在维度视角研究大语言模型中提示层面的标记表示几何结构。将Transformer视为均值场粒子系统，估算每层经验分布的内在维度，发现其与下一个词的不确定性相关。在不同模型和估计方法下，内在维度在早期至中期层达到峰值，并在词序打乱等语法语义破坏条件下上升；其与平均意外度（surprisal）强相关，简单分析表明对数几率几何可通过softmax关联熵。作为可解释性与安全性的案例研究，我们训练线性探测器基于逐层内在维度特征区分恶意与良性提示，在多个数据集上准确率达90%至95%，优于广泛使用的防护工具如Llama Guard和Shield Gemma。进一步对比基于Tuned Lens提取的逐层熵的探测器，发现内在维度探测器具有竞争力且互补，提供一种跨层分布、紧凑可解释的信号。结果表明，提示级别的几何结构可为监控与控制大模型行为提供可行动信号，连接机制理解与实际安全工具。
原文摘要 · Abstract (English)
We study the geometry of token representations at the prompt level in large language models through the lens of intrinsic dimension. Viewing transformers as mean-field particle systems, we estimate the intrinsic dimension of the empirical measure at each layer and demonstrate that it correlates with next-token uncertainty. Across models and intrinsic dimension estimators, we find that intrinsic dimension peaks in early to middle layers and increases under syntactic and semantic disruption (by shuffling tokens), and that it is strongly correlated with average surprisal, with a simple analysis linking logits geometry to entropy via softmax. As a case study in practical interpretability and safety, we train a linear probe on the per-layer intrinsic dimension profile to distinguish malicious from benign prompts before generation. This probe achieves accuracy of 90 to 95\% in different datasets, outperforming widely used guardrails such as Llama Guard and Shield Gemma. We further compare against linear probes built from layerwise entropy derived via the Tuned Lens and find that the intrinsic dimension-based probe is competitive and complementary, offering a compact, interpretable signal distributed across layers. Our findings suggest that prompt-level geometry provides actionable signals for monitoring and controlling LLM behavior, and offers a bridge between mechanistic insights and practical safety tools.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。