AI助手让安全分析师处理钓鱼邮件效率提升6.5倍，准确率提高77%。
Randomized Controlled Trials for Phishing Triage Agent
- 用随机对照试验评估AI助手在钓鱼邮件筛查中的表现
- 每分钟识别出的真阳性案例最多提升6.5倍，准确率提高77%
- 适合关注AI辅助决策与资源优化的安全团队负责人
安全运营中心（SOC）持续面临如何高效处理大量用户报告的钓鱼邮件，同时保持对威胁的有效防护。本文首次开展随机对照试验（RCT），评估领域专用AI助手——Microsoft Security Copilot钓鱼邮件筛查代理对分析师生产力和准确率的影响。结果表明，使用该代理的分析师每分钟可处理的真阳性邮件数量最高达控制组的6.5倍，判别准确率提升77%。代理的队列优先级排序与判定解释功能是效率提升的关键因素。行为分析显示，使用代理的分析师将注意力重新分配，对恶意邮件的处理时间增加53%，且未出现盲目采纳代理判断的情况。研究为考虑引入AI的安全主管提供了可操作的洞察，揭示了代理人可能从根本上改变SOC资源的最佳配置方式。
原文摘要 · Abstract (English)
Security operations centers (SOCs) face a persistent challenge: efficiently triaging a high volume of user-reported phishing emails while maintaining robust protection against threats. This paper presents the first randomized controlled trial (RCT) evaluating the impact of a domain-specific AI agent - the Microsoft Security Copilot Phishing Triage Agent - on analyst productivity and accuracy. Our results demonstrate that agent-augmented analysts achieved up to 6.5 times as many true positives per analyst minute and a 77% improvement in verdict accuracy compared to a control group. The agent's queue prioritization and verdict explanations were both significant drivers of efficiency. Behavioral analysis revealed that agent-augmented analysts reallocated their attention, spending 53% more time on malicious emails, and were not prone to rubber-stamping the agent's malicious verdicts. These findings offer actionable insights for SOC leaders considering AI adoption, including the potential for agents to fundamentally change the optimal allocation of SOC resources.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。