LLMs能模仿安全决策但不稳定,小改动就翻车。
Structured but Fragile: On the Limits of LLMs in Cybersecurity Decision-Making

- 用攻击图测试LLM选防御策略,结构清晰时表现接近最优解
- 图越复杂、提示词微调,结果波动大,最优策略常被低估
- 适合研究AI辅助安全决策的局限性,不推荐直接部署
大型语言模型(LLMs)在网络安全工作流中应用日益广泛,但其是否具备结构化安全推理能力仍不明确。本研究聚焦于真实威胁场景下的防御选择问题,涵盖勒索软件、供应链攻击、云滥用、Kubernetes攻击、POS恶意软件及工业控制系统/操作技术(ICS/OT)入侵等。在预算约束下,需从攻击图中选择安全控制以最小化攻击成功概率。我们比较了不同LLM策略与基于博弈论优化的基准方案,后者作为结构化推理的参考标准。结果表明,当提供明确的攻击图结构时,LLM常生成与基准相近的合理策略;然而其能力极为脆弱:随着图复杂度上升,表现显著下降,且对提示词表述高度敏感。微小的提示调整可大幅改变策略排序,仅将差策略标注为“最优”即可显著提升其评分。此外,形式化风险与LLM评估间存在非单调关系——最接近最优的策略未必获得最高评价。进一步测试发现,尽管LLM能生成正确高层逻辑的求解器,但其实际实现效率远低于专用求解器。总体而言,LLM可在受控表示下近似结构化安全推理,但缺乏鲁棒性,这对人工智能辅助安全决策系统的设计与评估具有重要意义。
原文摘要 · Abstract (English)
Large language models (LLMs) are increasingly used in cybersecurity workflows, yet it remains unclear whether they can perform structured security reasoning or merely rely on superficial cues and prior knowledge. We study this question in the context of defence selection over attack graphs derived from real-world threat scenarios, including ransomware, supply-chain compromise, cloud abuse, Kubernetes attacks, POS malware, and ICS/OT intrusion. Given a budget constraint, LLMs must select security controls to minimise attacker success. We compare their strategies against each other and against a game-theoretic optimization baseline used as a normative reference for structured reasoning. Our results show that LLMs exhibit conditional competence. When explicit attack-graph structure is provided, they often produce coherent strategies close to the optimization baseline. However, their capabilities are fragile. LLM behaviour becomes increasingly fragile with graph complexity and is highly sensitive to framing. Small prompt changes can substantially alter rankings, and merely relabeling a poor strategy as ``optimal'' dramatically improves its evaluation. We further observe a non-monotonic relationship between formal risk and LLM judgement: strategies closest to the optimum are not necessarily ranked highest by LLM evaluators. To further probe reasoning ability, we ask LLMs to generate solvers for the same optimization problem. While the generated implementations recover the correct high-level formulation, they scale poorly compared to a purpose-built solver. Overall, our findings show that LLMs can approximate structured cybersecurity reasoning under controlled representations, but do not apply it robustly. This has important implications for the design and evaluation of AI-assisted security decision-support systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。