arXiv:2503.10094cs.AIcs.CL2025-03被引 1

用智能技术从文本中自动提取技能,帮政策与教育决策提效。

Semantic Synergy: Unlocking Policy Insights and Learning Pathways Through Advanced Skill Mapping

  • 通过语义嵌入与搜索匹配,自动识别并关联多源文本中的技能
  • 显式技能检测F1超0.95,隐含提及准确率超0.93,接近人类水平
  • 适合政策制定、职业转型与终身学习领域从业者使用

本研究提出一个基于先进自然语言处理、语义嵌入与高效搜索技术的综合系统,用于从原始文本中挖掘相似性并生成可操作洞察。系统能自动提取并聚合来自政策文件、简历等多源文档的标准化能力信息,建立技能、职业画像与学习课程间的强关联。通过多层次评估验证性能,结果表明在合成与真实文档中,显式技能检测的F1得分超过0.95,隐含提及识别准确率高于0.93。方法采用多阶段流程:包括预处理、基于SentenceTransformer的语义嵌入与分段,以及基于FAISS的技能检索。提取的技能与ESCO职业框架及可持续发展目标学院提供的学习路径关联。配套的交互式可视化工具(基于Dash与Plotly)支持实时探索图表与表格,助力政策制定、培训供给、职业转换与招聘决策。该系统经严格验证,为提升政策制定、人力资源发展与终身学习提供了结构化且可行动的洞察。

原文摘要 · Abstract (English)

This research introduces a comprehensive system based on state-of-the-art natural language processing, semantic embedding, and efficient search techniques for retrieving similarities and thus generating actionable insights from raw textual information. The system automatically extracts and aggregates normalized competencies from multiple documents (such as policy files and curricula vitae) and creates strong relationships between recognized competencies, occupation profiles, and related learning courses. To validate its performance, we conducted a multi-tier evaluation that included both explicit and implicit skill references in synthetic and real-world documents. The results showed near-human-level accuracy, with F1 scores exceeding 0.95 for explicit skill detection and above 0.93 for implicit mentions. The system thereby establishes a sound foundation for supporting in-depth collaboration across the AE4RIA network. The methodology involves a multi-stage pipeline based on extensive preprocessing and data cleaning, semantic embedding and segmentation via SentenceTransformer, and skill extraction using a FAISS-based search method. The extracted skills are associated with occupation frameworks (as formulated in the ESCO ontology) and with learning paths offered through the Sustainable Development Goals Academy. Moreover, interactive visualization software, implemented with Dash and Plotly, presents graphs and tables for real-time exploration and informed decision-making by those involved in policymaking, training and learning supply, career transitions, and recruitment. Overall, this system, backed by rigorous validation, offers promising prospects for improved policymaking, human resource development, and lifelong learning by providing structured and actionable insights from raw, complex textual information.

技能映射智能决策终身学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。