arXiv:2505.23397cs.AIcs.CR2025-05被引 29

构建可信任的智能协作框架,提升安全中心人机协同效率

A Unified Framework for Human AI Collaboration in Security Operations Centers with Trusted Autonomy

  • 设计五级自主分级体系,匹配不同任务的人机角色与信任阈值
  • 在模拟攻防环境中验证,显著降低告警疲劳并提升响应协调性
  • 适合安全运营中心建设者、AI系统设计师参考,推动可信人机协作

本文提出一种结构化框架,用于安全运营中心(SOC)中的人工智能与人类协同,整合人工智能自主性、信任校准和人在回路中的决策机制。现有框架多聚焦自动化,缺乏系统性结构来管理人类监督、信任校准及可扩展的自主性。多数假设静态或二元的自主设置,未能考虑不同任务在复杂度、关键性和风险上的差异。为此,我们提出一个基于五级人工智能自主性(从人工到完全自主)的新型分层框架,对应特定任务的“人在回路”角色与信任阈值。该框架支持在监控、防护、威胁检测、告警甄别和事件响应等核心功能中实现自适应、可解释的AI集成。通过一个包含基于微调大模型的网络安全AI-Avatar的模拟攻防环境进行案例验证,结果表明该框架能有效减少告警疲劳,增强响应协调,并实现信任的策略性校准。研究系统阐述了下一代认知型安全中心的设计理论与实践可行性,强调AI应增强而非替代人类决策。

原文摘要 · Abstract (English)

This article presents a structured framework for Human-AI collaboration in Security Operations Centers (SOCs), integrating AI autonomy, trust calibration, and Human-in-the-loop decision making. Existing frameworks in SOCs often focus narrowly on automation, lacking systematic structures to manage human oversight, trust calibration, and scalable autonomy with AI. Many assume static or binary autonomy settings, failing to account for the varied complexity, criticality, and risk across SOC tasks considering Humans and AI collaboration. To address these limitations, we propose a novel autonomy tiered framework grounded in five levels of AI autonomy from manual to fully autonomous, mapped to Human-in-the-Loop (HITL) roles and task-specific trust thresholds. This enables adaptive and explainable AI integration across core SOC functions, including monitoring, protection, threat detection, alert triage, and incident response. The proposed framework differentiates itself from previous research by creating formal connections between autonomy, trust, and HITL across various SOC levels, which allows for adaptive task distribution according to operational complexity and associated risks. The framework is exemplified through a simulated cyber range that features the cybersecurity AI-Avatar, a fine-tuned LLM-based SOC assistant. The AI-Avatar case study illustrates human-AI collaboration for SOC tasks, reducing alert fatigue, enhancing response coordination, and strategically calibrating trust. This research systematically presents both the theoretical and practical aspects and feasibility of designing next-generation cognitive SOCs that leverage AI not to replace but to enhance human decision-making.

人机协同安全运营可信自治

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。