用强化学习训练大模型自动生成网络过滤规则,快速防御漏洞攻击。
REFN: A Reinforcement-Learning-From-Network Framework against 1-day/n-day Exploitations
- 通过在线网络奖励驱动的强化学习,让大模型自主生成防御规则。
- 在22类漏洞上实现94.3%准确率,修复时间缩短至3.65小时。
- 适合需要快速响应漏洞的网络安全团队和边缘设备防护场景。
1天或n天漏洞的利用对联网设备构成严重威胁,主要因部署规模大且补丁延迟(平均修复时间超过60天)。现有防御手段如主机补丁和网络过滤,受限于设备多样性下的可扩展性差、兼容性问题(尤其嵌入式或旧系统)以及部署过程易出错(需手动验证补丁)。为此,我们提出REFN(基于网络的强化学习),一种新型框架,通过训练大型语言模型(LLMs)自动生成网络过滤规则以防止1天或n天漏洞被利用。REFN通过在线网络奖励驱动的强化学习(而非传统人类反馈强化学习),确保可扩展性;通过统一部署在边缘安全网关(Amazon Eero)保障兼容性;通过实时网络流量进行在线验证提升鲁棒性。关键突破包括:1)通过基于代理RAG的知识蒸馏扩展模型漏洞修复能力;2)通过从VNF管道将漏洞描述语言转化为网络执行策略,弥合语言与网络间的鸿沟;3)通过在线代理验证机制惩罚错误输出,缓解幻觉与非确定性问题。在22类1天或n天漏洞上评估显示,REFN相较其他方法准确率提高21.1%,平均修复时间仅3.65小时,可轻松扩展至10,000台设备。REFN为训练大模型快速应对大规模漏洞攻击迈出关键一步。
原文摘要 · Abstract (English)
The exploitation of 1 day or n day vulnerabilities poses severe threats to networked devices due to massive deployment scales and delayed patching (average Mean Time To Patch exceeds 60 days). Existing defenses, including host based patching and network based filtering, are inadequate due to limited scalability across diverse devices, compatibility issues especially with embedded or legacy systems, and error prone deployment process (manual patch validation). To address these issues, we introduce REFN (Reinforcement Learning From Network), a novel framework that trains Large Language Models (LLMs) to autonomously generate network filters to prevent 1 day or n day exploitations. REFN ensures scalability by uniquely employs Reinforcement Learning (RL) driven by online network rewards instead of traditional Human Feedback (RLHF). REFN guarantees compatibility via unified deployment on edge security gateways (Amazon Eero). REFN provides robustness via online validation using real network traffic. Crucially, REFN addresses three core challenges in training LLMs for exploit prevention: 1) expanding current LLMs limited vulnerability fixing expertise via Agentic RAG based Knowledge Distillation, 2) bridging current LLMs language to network gaps through an RL From VNF Pipeline that translates language context (vulnerability description) into network enforcement, 3) addressing the LLM hallucination and non determinism via the Online Agentic Validation that penalizes erroneous outputs. Evaluated across 22 families of 1 day or n day exploits, REFN demonstrates effectiveness (21.1 percent higher accuracy than alternatives), efficiency (Mean Time To Patch of 3.65 hours) and scalability (easily scale to 10K devices). REFN serves as an initial step toward training LLMs to rapidly prevent massive scale 1 day or n day exploitations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。