arXiv:2608.06778cs.CRcs.CL2026-08中稿 · the AI Agent for I…

用检索+强化学习提升威胁情报技术提取的准确率与速度

Retrieval-Constrained Policy Optimization for Attack Technique Extraction from Cyber Threat Intelligence

论文配图:Retrieval-Constrained Policy Optimization for Attack Technique Extraction from Cyber Threat Intelligence
图 1 · 摘自论文原文
  • 先检索候选技术集,再用微调大模型选择正确项
  • 在四个数据集上平均F1最优,子技术识别提升7.4个百分点
  • 适合需要高效精准提取攻击技术的网络安全研究人员

将网络威胁情报(CTI)文本映射到MITRE ATT&CK技术是结构化威胁分析的关键,但人工标注成本高且难扩展。ATT&CK包含数百个攻击技术,单段CTI可能描述多个技术,导致准确完整提取困难。现有自动化方法存在不足:多标签分类器受严重类别不平衡和大标签空间影响;基于大模型的方法(检索流水线与微调生成器)优化的是词元级目标,将技术标注视为序列生成而非集合预测,缺乏对预测集合是否正确完整的直接监督。我们提出TTP-R1,一种两阶段框架,结合检索增强的监督微调(SFT)与可验证奖励的强化学习(RLVR)。混合检索器先将大标签空间缩减为候选集,微调的大模型学习选择正确技术。随后采用组相对策略优化,使用分解奖励直接监督预测集合的精确率、召回率与输出格式。在四个CTI基准上,TTP-R1实现最佳平均F1,子技术层面比带检索增强的Claude Sonnet 4.5提升7.4个百分点,且作为80亿参数模型在单张GPU上运行速度快28倍。

原文摘要 · Abstract (English)

Mapping cyber threat intelligence (CTI) text to MITRE ATT&CK techniques is essential for structured threat analysis, yet manual annotation is costly and does not scale. The ATT&CK taxonomy comprises several hundred attack techniques, and a single CTI passage may describe multiple techniques, making accurate and complete extraction challenging. Existing automated approaches fall short in different ways: multi-label classifiers struggle with severe class imbalance and the large label space, while LLM-based methods--retrieval pipelines and fine-tuned generators--optimize token-level objectives that treat technique annotation as sequence generation rather than set prediction, lacking direct supervision on whether the predicted technique set is correct and complete. We propose TTP-R1, a two-stage framework that combines retrieval-augmented supervised fine-tuning (SFT) with reinforcement learning using verifiable rewards (RLVR). A hybrid retriever first narrows the large label space to a candidate set, and a fine-tuned LLM learns to select the correct techniques. We then apply Group Relative Policy Optimization with a decomposed reward that directly supervises the precision, recall, and output format of the predicted technique set. Across four CTI benchmarks, TTP-R1 achieves the best average F1, improving sub-technique-level F1 by 7.4 percentage points over Claude Sonnet 4.5 with retrieval augmentation, while running 28x faster when served as an 8B-parameter model on a single GPU.

威胁情报大模型应用强化学习ATT&CK

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。