arXiv:2608.10216cs.CLcs.AI2026-08

用嵌入余弦相似度判断语义是否改变,结果常适得其反。

Similarity Gates Approve Reversals: A Validity Audit of Embedding-Cosine Thresholds in Agent Systems

  • 以固定阈值判断文本语义是否一致,但实际测量的是措辞变化程度。
  • 审计显示安全门在56次语义破坏中0次捕获,反而通过了关键反转例。
  • 现有方法易受混淆影响,建议改用配对验证等更可靠评估方式。

代理系统使用基于嵌入余弦相似度的质量检查门控机制,判断文本是否保持原意。但该指标实际衡量的是表述变化程度,而非语义一致性。当指令反转仅需微调一个词时,语义变更却可能被大量重述掩盖,导致检测失效。审计发现,生产环境的漂移防护机制在56个语义破坏样本中未捕捉任何异常,而一句关键反转(' withhold the study drug' → ' administer the study drug')仍以0.9608的余弦相似度通过。九种配置下平衡准确率最高仅0.700(中位数0.525)。同一批数据因混淆效应导致评估结果倒置:13/18配置细胞中AUROC为0.000,最大仅0.040,远低于平衡设计下的0.440–0.815。替换编码器、引入重叠条件门或使用NLI模型均未能提升效果。仅有两种配置在匹配重叠条件下能有效区分反转与改写(AUROC 0.79–0.90),且必须依赖配对审计。研究发布数据集、工具和冻结结果,主张当前门控机制度量错误,但认为可构建有效仪器。

原文摘要 · Abstract (English)

Agent frameworks ship quality gates that compare text blocks by embedding-cosine similarity and decide at a fixed cutoff. Deduplication filters, semantic caches, drift guards, and answer grader gates deploy to answer the question: "Does this text still mean the same thing?" But the score answers a different question: "How much did the wording change?" We audit this gate class as a measurement instrument. In the cases these gates exist to catch, the two can run in opposite ways. Many times, reversing an instruction is a single word edit, while agreement often rephrases a sentence. The consequence is a safety check that fires backwards. The production drift guard we audited caught 0 of 56 meaning-breaking mutations, and one approved item, "withhold the study drug" -> "administer the study drug", came in at cosine 0.9608. We observed five shipped operating points, and balanced accuracy across 90 configuration-threshold-task cells never exceeded 0.700 (median 0.525). The same confounder also corrupted evaluations. A naively built corpus inherits this confounder and can return an inverted verdict, with a decision AUROC exactly 0.000 in 13 of 18 configuration-task cells (at most 0.040 in all 18) against 0.440-0.815 for the same nine configurations under a balanced 2x2 design. Twice in the effort it captured our own headline claims. Obvious repairs fail: an encoder swap and an overlap-conditioned gate (0.750 in-sample, 0.533 held-out) land at chance on separately authored held-out data, and an NLI drop-in did no better. Embeddings do still bear hope here, as the strongest two of nine configurations separated reversal from paraphrase at matched overlap (AUROC 0.79-0.90), but only a matched-pair audit reveals the deployment regime. We release the corpus method, harness, and frozen results, and contend that scores gated this way measure the wrong thing. We believe a valid instrument is buildable.

语义检测安全门控嵌入评估偏差审计

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。