arXiv:2605.24663cs.CRcs.AI2026-05

用专家反馈优化关键词匹配,提升网络安全课程知识映射准确率

CyBOKClaw: Human-in-the-Loop CyBOK Mapping for Cybersecurity Curriculum

  • 人机协同检索框架,生成候选映射供专家筛选
  • 验证集上98%的候选结果含专家认可的近似匹配
  • 适合教育者构建或更新网络安全课程体系

本文提出CyBOKClaw,一种可解释的人机协同检索框架,用于将网络安全关键词或短语(KWoPs)映射到网络安全知识体系(CyBOK)。不同于严格精确匹配,该框架设计为专家评审用的前k名候选生成器。其结合查询归一化、术语扩展、概念级加权、主题描述增强及领域敏感排序规则。由于教育类关键词常具宽泛性、模糊性且与CyBOK术语仅近似对应,严格精确匹配难以反映实际价值。因此,采用结构检索指标与专家指导的前5名实用性指标ECA-5(前5名中存在专家认定的精确或最近似可接受匹配)进行评估。在开发数据集上,实现64.73%的EXA-5、84.18%结构语义对齐和91.88% ECA-5;在验证数据集上,达到81.19% EXA-5、93.32%结构语义对齐和98.00% ECA-5。结果表明,专家导向的前k名实用性评估比单一精确结构匹配更真实反映实际映射价值,且CyBOKClaw作为面向CyBOK的专家辅助检索系统有效。

原文摘要 · Abstract (English)

This paper presents CyBOKClaw, an interpretable human-in-the-loop retrieval framework for mapping cybersecurity keywords or phrases (KWoPs) to the Cyber Security Body of Knowledge (CyBOK). Rather than treating the task as strict exact classification, the framework is designed as a top-k candidate generator for expert review. It combines query normalization, curated term expansion, concept-level boosts, topic-description enrichment, and domain-sensitive ranking rules. Because educational KWoPs are often broad, ambiguous, and only approximately aligned with CyBOK terminology, strict exact matching provides only a partial account of practical utility. We therefore evaluate the framework using both structural retrieval metrics and an expert-guided top-5 usefulness metric, ECA-5 (Exact or Closest Acceptable Match at top-5), which records whether the returned candidates contain at least one mapping that an expert would judge exact or accept as the nearest practical CyBOK placement. On the development dataset, CyBOKClaw achieves 64.73% EXA-5 (Exact Match at top-5), 84.18% structural semantic alignment, and 91.88% ECA-5; on the validation dataset, it achieves 81.19% EXA-5, 93.32% structural semantic alignment, and 98.00% ECA-5. These results show that expert-guided top-k usefulness provides a more faithful account of practical CyBOK mapping utility than exact structural matching alone, and that CyBOKClaw is effective as a CyBOK-specific expert-support retrieval system.

知识映射网络安全教育系统人机协同

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。