arXiv:2512.02318cs.CRcs.AI2025-12中稿 · USENIX Sec'26被引 3

多模态大模型能低成本破解多数验证码,但对精细定位类任务仍难奏效。

COGNITION: From Evaluation to Defense against Multimodal LLM CAPTCHA Solvers

  • 用7个主流模型测试18类真实验证码,评估准确率与成本
  • 95%以上成功率的验证码经结构优化后可降至0%
  • 适合安全研究者参考,指导下一代验证码设计

本文研究多模态大语言模型(MLLMs)如何威胁视觉验证码的安全性。我们识别出攻击面:攻击者可低成本使用现成模型自动化破解验证码。在18种真实世界验证码任务上评估7个代表性MLLMs,测量单次准确率、有限重试下的成功率、端到端延迟及每解题成本。通过补充外部数据集和带会话记忆的自适应攻击者设置进一步验证,同时分析任务特异性提示工程与少样本演示对求解效果的影响。结果表明,当前模型可在人类级成本与延迟下可靠解决以识别为主的低交互验证码,但对精细定位、多步空间推理或跨帧一致性要求高的任务仍显著困难。通过分析模型推理轨迹,揭示其在特定验证码上成功/失败机制,并据此提出防御导向的设计指南。为验证原则,我们基于指南加固一种易受攻击的验证码类型,结果显示状态最先进模型的成功率从超过95%降至0%,证明结构改进可有效缓解威胁。结论强调需紧急重构验证码以应对日益增强的MLLM能力。代码已公开(https://doi.org/10.5281/zenodo.20406852)。

原文摘要 · Abstract (English)

This paper studies how multimodal large language models (MLLMs) undermine the security guarantees of visual CAPTCHA. We identify the attack surface where an adversary can cheaply automate CAPTCHA solving using off-the-shelf models. We evaluate 7 representative MLLMs on 18 real-world CAPTCHA task types, measuring single-shot accuracy, success under limited retries, end-to-end latency, and per-solve cost. We further validate our findings through a supplemental external dataset and an adaptive-attacker setting with session memory, while also analyzing the impact of task-specific prompt engineering and few-shot demonstrations on solver effectiveness. We reveal that MLLMs can reliably solve recognition-oriented and low-interaction CAPTCHA tasks at human-like cost and latency, whereas tasks requiring fine-grained localization, multi-step spatial reasoning, or cross-frame consistency remain significantly harder for current models. By examining the reasoning traces of such MLLMs, we investigate the underlying mechanisms of why models succeed/fail on specific CAPTCHA puzzles and use these insights to derive defense-oriented guidelines for selecting and strengthening CAPTCHA tasks. To validate these principles, we present a proof-of-concept by hardening a vulnerable CAPTCHA type using our guidelines. We demonstrate that incorporating fine-grained localization and implicit counting reduces the success rate of state-of-the-art MLLMs from over 95\% to 0\%, confirming that structural changes can effectively mitigate the threat. We conclude by emphasizing the urgent need for CAPTCHA redesign as MLLM capabilities increasingly threaten existing defenses. Code Availability (https://doi.org/10.5281/zenodo.20406852).

验证码安全多模态模型对抗攻击防御设计

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。