用梯度搜索找让大模型出错的异常输入，无需依赖特定模型结构。
GlitchMiner: Mining Glitch Tokens in Large Language Models via Gradient-based Discrete Optimization
- 通过最大化预测熵，用梯度指导的局部搜索寻找异常令牌。
- 在10个不同架构的大模型上检测准确率和查询效率均优于现有方法。
- 不依赖模型特异性规则，适合各类大模型的安全性测试。
Glitch tokens 是指引发大语言模型（LLMs）不可预测或异常行为的输入，严重威胁模型的可靠性与安全性。现有检测方法主要依赖于启发式嵌入模式或内部表示中的统计异常，限制了其在不同模型架构间的泛化能力，且可能遗漏偏离已有模式的异常。我们提出 GlitchMiner，一种以行为为导向的框架，通过最大化预测熵来识别 glitch tokens。该方法采用梯度引导的局部搜索策略，在不依赖模型特定启发式或大批次采样的前提下，高效探索离散的令牌空间。在来自五个主要模型家族的十个 LLM 上的大量实验表明，GlitchMiner 在检测准确率和查询效率方面持续优于现有方法，提供了一种通用且可扩展的 glitch token 发现方案。代码已公开于 [https://github.com/wooozihu/GlitchMiner]。
原文摘要 · Abstract (English)
Glitch tokens, inputs that trigger unpredictable or anomalous behavior in Large Language Models (LLMs), pose significant challenges to model reliability and safety. Existing detection methods primarily rely on heuristic embedding patterns or statistical anomalies within internal representations, limiting their generalizability across different model architectures and potentially missing anomalies that deviate from observed patterns. We introduce GlitchMiner, an behavior-driven framework designed to identify glitch tokens by maximizing predictive entropy. Leveraging a gradient-guided local search strategy, GlitchMiner efficiently explores the discrete token space without relying on model-specific heuristics or large-batch sampling. Extensive experiments across ten LLMs from five major model families demonstrate that GlitchMiner consistently outperforms existing approaches in detection accuracy and query efficiency, providing a generalizable and scalable solution for effective glitch token discovery. Code is available at [https://github.com/wooozihu/GlitchMiner]
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。