arXiv:2510.17884cs.CRcs.AI2025-10被引 1

大模型生成密码准确率不足1.5%,远不如传统方法。

When Intelligence Fails: An Empirical Study on Why LLMs Struggle with Password Cracking

  • 用用户属性提示大模型生成密码,测试其推理能力。
  • 所有模型在Hit@10下准确率均低于1.5%。
  • 适合研究大模型安全边界与密码建模的局限性。

大型语言模型(LLMs)在自然语言理解与生成方面的卓越能力引发了其在网络安全应用中的兴趣,包括密码猜测。本研究通过合成用户资料,对预训练开源大模型(如TinyLLaMA、Falcon-RW-1B、Flan-T5)进行实证评估,要求其根据姓名、出生日期、爱好等结构化用户属性生成合理密码。使用命中率(Hit@1、Hit@5、Hit@10)在明文和SHA-256哈希对比下衡量性能,结果显示所有模型在Hit@10下的准确率均低于1.5%。相比之下,传统规则与组合式破解方法表现显著更优。通过详细分析与可视化,我们揭示了大模型在密码推断这一特定任务中生成推理的关键缺陷。研究发现,尽管具备语言能力,当前大模型仍缺乏领域适配与记忆能力,尤其在未经过泄露密码数据集监督微调时,难以有效进行密码推断。该研究为理解大模型在对抗场景中的局限性提供了关键洞见,并为未来安全、隐私保护且鲁棒的密码建模奠定了基础。

原文摘要 · Abstract (English)

The remarkable capabilities of Large Language Models (LLMs) in natural language understanding and generation have sparked interest in their potential for cybersecurity applications, including password guessing. In this study, we conduct an empirical investigation into the efficacy of pre-trained LLMs for password cracking using synthetic user profiles. Specifically, we evaluate the performance of state-of-the-art open-source LLMs such as TinyLLaMA, Falcon-RW-1B, and Flan-T5 by prompting them to generate plausible passwords based on structured user attributes (e.g., name, birthdate, hobbies). Our results, measured using Hit@1, Hit@5, and Hit@10 metrics under both plaintext and SHA-256 hash comparisons, reveal consistently poor performance, with all models achieving less than 1.5% accuracy at Hit@10. In contrast, traditional rule-based and combinator-based cracking methods demonstrate significantly higher success rates. Through detailed analysis and visualization, we identify key limitations in the generative reasoning of LLMs when applied to the domain-specific task of password guessing. Our findings suggest that, despite their linguistic prowess, current LLMs lack the domain adaptation and memorization capabilities required for effective password inference, especially in the absence of supervised fine-tuning on leaked password datasets. This study provides critical insights into the limitations of LLMs in adversarial contexts and lays the groundwork for future efforts in secure, privacy-preserving, and robust password modeling.

大模型安全密码破解生成推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。