检测大模型推理中被恶意消耗令牌的攻击,提前终止异常生成。
RecurGuard: Runtime Monitoring for Reasoning-Token Consumption Attacks
- 通过分析推理过程中的重复率、增长量和问题进展三信号实时监控。
- 在7B模型上对99%的OverThink攻击和92%的ExtendAttack有效检测。
- 适合部署于需防护令牌滥用的AI服务场景,如收费对话系统。
具备推理能力的大语言模型可能被诱导将生成预算用于注入的干扰任务,导致无法输出最终答案(拒绝服务)或产生超额计费(钱包拒绝)。输入侧安全分类器常因注入提示语法上看似正常而漏检。我们构建了RecurGuard,一种在推理轨迹暴露时运行的实时监控系统。RecurGuard在生成过程中分析推理轨迹,追踪三个信号:重复率、体积增长和向用户问题的进展。若这三个信号在连续三个片段中均异常,系统将提前终止生成。我们在开源推理模型上评估了RecurGuard对OverThink和ExtendAttack的防御效果,并对DS-R1-Qwen-7B进行了自适应压力测试。结果显示,该系统在该模型上对99%的OverThink攻击和92%的ExtendAttack实现有效检测,同时在问答、代码生成、数学和摘要任务中保持接近零的误报率。自适应评估揭示防御极限:主题型攻击仍保留11.9倍放大效应,联合误检率达约50%;而完整语义逃避可使放大从22.8倍降至2.2倍。当推理轨迹不可用时,QDM提供基于最终输出的后置监控作为备用方案。
原文摘要 · Abstract (English)
Reasoning-capable large language models can be induced to spend their generation budget on injected decoy tasks rather than answering the user's question, causing denial of service when no final answer is produced and denial of wallet when excess output tokens are billed. Input-side safety classifiers often miss these attacks because the injected prompts can appear syntactically benign. We build RecurGuard, a runtime monitor for detecting reasoning-chain consumption attacks when reasoning traces are exposed by the model. RecurGuard analyzes reasoning traces as they are generated and tracks three signals: recurrence rate, volume growth, and progress toward the user's query. If all three signals remain anomalous over three consecutive chunks, RecurGuard terminates generation early. We evaluate RecurGuard against OverThink and ExtendAttack across open-weight reasoning models and conduct adaptive stress tests on DS-R1-Qwen-7B. On this model, RecurGuard detects 99% of OverThink attacks and 92% of ExtendAttack instances while maintaining near-zero false positive rates on question answering, code generation, mathematics, and summarization. Adaptive evaluation reveals the limit of the defense: topical attacks retain 11.9x amplification with an approximately 50% joint miss rate, whereas full semantic evasion reduces amplification from 22.8x to 2.2x. When reasoning traces are unavailable, QDM provides a post-hoc fallback monitor based on the final output.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。