测试大模型生成密码策略的可靠性和一致性。
How Good LLM-Generated Password Policies Are?
- 用自然语言提示让大模型直接生成配置文件。
- 对比有无官方文档指导时生成结果的准确度差异。
- 揭示当前大模型在安全配置生成中的可靠性问题。
生成式AI技术，尤其是大型语言模型（LLMs），因其在自然语言处理方面的卓越能力，正迅速被工业界、学术界和政府部门采纳。然而，尽管具备强大能力，LLMs输出的不一致性和不可预测性在安全关键领域（如访问控制）带来了显著挑战。本文研究了LLMs在网络安全访问控制系统中的应用，重点考察其生成密码策略的准确性与一致性，将自然语言提示转换为可执行的pwquality.conf配置文件。实验采用两种方法：一是仅凭自然语言提示生成配置文件；二是提供官方pwquality.conf文档作为参考基准。我们系统评估了这些由AI生成配置的合理性、准确性和一致性。研究发现，当前大模型在该任务中存在严重缺陷，为优化大模型在访问控制系统中的部署提供了重要洞见。
原文摘要 · Abstract (English)
Generative AI technologies, particularly Large Language Models (LLMs), are rapidly being adopted across industry, academia, and government sectors, owing to their remarkable capabilities in natural language processing. However, despite their strengths, the inconsistency and unpredictability of LLM outputs present substantial challenges, especially in security-critical domains such as access control. One critical issue that emerges prominently is the consistency of LLM-generated responses, which is paramount for ensuring secure and reliable operations. In this paper, we study the application of LLMs within the context of Cybersecurity Access Control Systems. Specifically, we investigate the consistency and accuracy of LLM-generated password policies, translating natural language prompts into executable pwquality$.$conf configuration files. Our experimental methodology adopts two distinct approaches: firstly, we utilize pre-trained LLMs to generate configuration files purely from natural language prompts without additional guidance. Secondly, we provide these models with official pwquality$.$conf documentation to serve as an informative baseline. We systematically assess the soundness, accuracy, and consistency of these AI-generated configurations. Our findings underscore significant challenges in the current generation of LLMs and contribute valuable insights into refining the deployment of LLMs in Access Control Systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。