让大模型学会像专家一样推理安全告警真伪，减少误判漏判。
Cybersecurity Detection Classification with Reasoning-enabled Language Models
- 用思维链+自我训练+强化学习，让模型学会分析告警是否真实威胁。
- 在高置信度下，误报召回率提升43%，恶意告警召回率提升18.3%。
- 专门训练的30B模型优于通用大模型，适合安全运维场景使用。
安全运营中心面临告警疲劳问题，每日检测数量远超人工处理能力。现有方法直接让大模型输出分类标签，但缺乏对告警真实性的推理能力。本文在真实、人工标注的Windows终端检测数据上，结合自动提示优化、自训练和可验证奖励的强化学习，训练了具备思维链（CoT）推理能力的告警分类器。发现思维链会降低标签概率，因此额外训练了一个校准器，读取完整推理过程并估计判断正确概率。系统测试准确率达82.6%，在高置信度操作点上，相比直接输出标签的LLM分类器，良性告警召回率提升43.0%，恶意告警召回率提升18.3%。实验表明校准器不可或缺，未训练的置信度判断会使高置信度召回归零；同时，微调后的30B模型显著优于前沿通用大模型，证明针对性训练比单纯扩大规模更有效。
原文摘要 · Abstract (English)
A major issue in Security Operations Centers (SOCs) is alert fatigue, as the number of detections reported is more than staff can triage in a given day. Prior work prompts or fine-tunes large language models (LLMs) to emit a triage label directly, but does not train them to reason about whether a detection is a genuine threat. We train a chain-of-thought (CoT) reasoning-enabled triage classifier on real, human-labeled Windows endpoint detections by combining automated prompt optimization, self-training, and reinforcement learning with verifiable rewards. We find that CoT reasoning also degrades the label-token probabilities that automated triage relies on, so we separately train a calibrator that reads the full reasoning trace and estimates the probability that the verdict is correct. Our system reaches 82.6% test accuracy and, at the high-confidence operating point that governs automated triage, improves benign recall by 43.0% and malicious recall by 18.3% over a direct-label LLM classifier. We further show that the trained calibrator is necessary - an untrained confidence judge collapses high-confidence recall to zero - and that a finetuned 30B model significantly outperforms frontier general-purpose models, motivating targeted training over scale.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。