剖析深度强化学习在网络安全中的11个常见陷阱，警示研究者避免无效实验。
SoK: The Pitfalls of Deep Reinforcement Learning for Cybersecurity
- 系统梳理从建模到部署各阶段的11类方法缺陷
- 分析66篇论文发现平均每篇含超5个陷阱
- 提供可操作建议，助构建更可靠的网络安全AI
深度强化学习（DRL）在序列决策任务中表现优异，推动其在网络安全领域的应用。然而，将DRL从实验室模拟迁移到真实网络环境时，常因安全任务具有对抗性、非平稳性和部分可观测性而引入诸多问题。本文系统识别并归纳了DRL用于网络安全（DRL4Sec）研究中，在环境建模、智能体训练、性能评估和系统部署四个阶段常见的11类方法论陷阱。通过对2018至2025年间66篇重要DRL4Sec论文的分析，量化每类陷阱的普遍性，发现平均每篇论文存在超过五个陷阱。通过在自主防御、恶意软件生成和网页安全测试三个场景中开展受控实验，验证了这些陷阱的实际影响。最后，针对每一类陷阱提出具体改进建议，以支持开发更严谨且可部署的DRL安全系统。
原文摘要 · Abstract (English)
Deep Reinforcement Learning (DRL) has achieved remarkable success in domains requiring sequential decision-making, motivating its application to cybersecurity problems. However, transitioning DRL from laboratory simulations to bespoke cyber environments can introduce numerous issues. This is further exacerbated by the often adversarial, non-stationary, and partially-observable nature of most cybersecurity tasks. In this paper, we identify and systematize 11 methodological pitfalls that frequently occur in DRL for cybersecurity (DRL4Sec) literature across the stages of environment modeling, agent training, performance evaluation, and system deployment. By analyzing 66 significant DRL4Sec papers (2018-2025), we quantify the prevalence of each pitfall and find an average of over five pitfalls per paper. We demonstrate the practical impact of these pitfalls using controlled experiments in (i) autonomous cyber defense, (ii) adversarial malware creation, and (iii) web security testing environments. Finally, we provide actionable recommendations for each pitfall to support the development of more rigorous and deployable DRL-based security systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。