提出诊断框架,识别大模型分类器决策的可信度与漏洞。
The Latent Diagnostic Taxonomy: A Framework for Constructing Classifiers and Diagnosing Their Decisions, Applied to Prompt Injection Detection
- 动态优化嵌入维度,筛选关键提示样本以定位漏洞
- 77%自信判断受单个词影响,分出两类失效模式
- 提供分级处理策略,适合安全检测与模型调试
本文提出一种名为潜在诊断分类法(Latent Diagnostic Taxonomy)的框架,用于构建防护性分类器并诊断其决策可靠性。该框架包含三步:(i) 通过交叉验证选择最优嵌入维度,而非预先固定;(ii) 找到约29%的潜在支持向量,代表影响分类结果的关键提示片段;(iii) 利用这些片段及其攻击强度构建诊断分类体系。该体系可对提示进行分级处理:信任分类器决策、标记启发式偏差或覆盖情况、将上下文不足的提示交由人工或安全审查。在公开提示注入数据集上的实验表明,约77%的分类器自信判断在移除单个词后即失效,且此类脆弱性分为两类:置信度校准失败与真实可被利用的捷径。针对每类问题,均给出修复建议,并展示框架各步骤的完整运行流程。
原文摘要 · Abstract (English)
This paper proposes a framework for constructing a classifier as a safeguard layer, and for developing a complementary diagnostic that identifies which of the classifier's confident decisions can be trusted. This framework, the Latent Diagnostic Taxonomy, consists of (i) constructing a dimensionality-optimized classifier, in which the embedding dimensionality is empirically selected via cross-validated performance rather than fixed a priori, (ii) locating a relatively small set of latent support vectors (~ 29% of total training examples) representing influential prompts for identifying tokens that alter the classifier's predicted labels, and (iii) utilizing such tokens and their associated attack magnitudes for constructing a diagnostic taxonomy. This diagnostic taxonomy provides an end-to-end guideline for flagging prompts that require different treatments: rely Safely on the classifier's decision; flag Heuristic Bias and Heuristic Override cases; route Insufficient Context cases for further human/safety review. Applying the framework to a classifier trained on a public prompt injection dataset, we find that a substantial fraction of its confident decisions (~ 77%) are not robust to removing a single token, and that this brittleness separates into two distinct failure patterns: a confidence calibration failure and a genuinely exploitable shortcut. For each zone of the taxonomy, we also recommend strategies for remediating diagnosed prompts. We illustrate the framework as a series of steps, demonstrating how each step operates.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。