arXiv:2503.02065cs.CRcs.AI2025-03被引 7

研究安全团队如何信任AI解释,发现上下文丰富的说明能提升判断效率。

Too Much to Trust? Measuring the Security and Cognitive Impacts of Explainability in AI-Driven SOCs

  • 通过问卷和访谈分析分析师对AI解释的期待
  • 有证据支撑的解释即使准确率低也更可信
  • 针对不同角色设计上下文型解释更实用

可解释AI(XAI)在安全运营中心(SOCs)中有望提升AI威胁检测的透明度与可信度。然而,在高压力、快决策环境下,如何确定合适的解释程度与形式仍是未充分探索的难题。本研究开展为期三个月的混合方法研究,结合在线调查(N1=248)与深度访谈(N2=24),探究(1)分析师如何理解AI生成的解释,(2)哪些类型的解释在不同岗位中被认为可操作且可信。结果表明,当解释被视作相关且有证据支持时,参与者即使面对较低预测准确率也愿接受。分析师反复强调理解AI决策依据的重要性,偏好包含上下文深度的说明,而非仅展示结果的仪表板。基于此,本文重新评估了现有解释方法,并证明:与SOC工作流程匹配、角色感知、内容丰富的XAI设计能显著提升实用性,增强分析师理解力,提高告警处理效率,支持更自信地应对动态威胁。

原文摘要 · Abstract (English)

Explainable AI (XAI) holds significant promise for enhancing the transparency and trustworthiness of AI-driven threat detection in Security Operations Centers (SOCs). However, identifying the appropriate level and format of explanation, particularly in environments that demand rapid decision-making under high-stakes conditions, remains a complex and underexplored challenge. To address this gap, we conducted a three-month mixed-methods study combining an online survey (N1=248) with in-depth interviews (N2=24) to examine (1) how SOC analysts conceptualize AI-generated explanations and (2) which types of explanations are perceived as actionable and trustworthy across different analyst roles. Our findings reveal that participants were consistently willing to accept XAI outputs, even in cases of lower predictive accuracy, when explanations were perceived as relevant and evidence-backed. Analysts repeatedly emphasized the importance of understanding the rationale behind AI decisions, expressing a strong preference for contextual depth over a mere presentation of outcomes on dashboards. Building on these insights, this study re-evaluates current explanation methods within security contexts and demonstrates that role-aware, context-rich XAI designs aligned with SOC workflows can substantially improve practical utility. Such tailored explainability enhances analyst comprehension, increases triage efficiency, and supports more confident responses to evolving threats.

可解释AI安全运营人机信任认知负荷

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。