首个土耳其语技能抽取数据集,用大模型提升低资源语言招聘匹配效率
Leveraging LLMs For Turkish Skill Extraction
- 构建首个土耳其语技能抽取数据集,含4819个标注技能片段
- 大模型端到端方法在ESCO标准对齐上达0.56准确率
- 动态提示+检索重排序策略显著提升低资源语言性能
技能抽取是现代招聘系统的关键环节,有助于高效职位匹配、个性化推荐和劳动力市场分析。尽管土耳其在全球劳动力中占重要地位,但其形态复杂的语言缺乏技能分类体系和专用技能抽取数据集,导致相关研究严重不足。本文针对三个问题展开:1)如何在资源匮乏条件下有效进行土耳其语技能抽取?2)哪种模型表现最佳?3)不同大语言模型(LLMs)及提示策略(如动态/静态少样本示例、上下文信息变化、鼓励因果推理)的影响如何?本文首次提出土耳其语技能抽取数据集,并评估了基于LLM的自动化技能抽取性能。该人工标注数据集包含来自327份职位广告的4,819个标注技能片段,覆盖多个职业领域。实验表明,将LLM用于端到端流程,相比传统监督序列标注,在与ESCO分类标准对齐方面表现更优。最佳配置为使用Claude Sonnet 3.7,结合动态少样本提示、基于嵌入的检索以及LLM重排序进行技能关联,实现0.56的端到端性能,达到其他语言类似研究的水平。结果表明,大模型可有效提升低资源语言的技能抽取效果,期望推动更多关于非主流语言技能抽取的研究。
原文摘要 · Abstract (English)
Skill extraction is a critical component of modern recruitment systems, enabling efficient job matching, personalized recommendations, and labor market analysis. Despite Türkiye's significant role in the global workforce, Turkish, a morphologically complex language, lacks both a skill taxonomy and a dedicated skill extraction dataset, resulting in underexplored research in skill extraction for Turkish. This article seeks the answers to three research questions: 1) How can skill extraction be effectively performed for this language, in light of its low resource nature? 2)~What is the most promising model? 3) What is the impact of different Large Language Models (LLMs) and prompting strategies on skill extraction (i.e., dynamic vs. static few-shot samples, varying context information, and encouraging causal reasoning)? The article introduces the first Turkish skill extraction dataset and performance evaluations of automated skill extraction using LLMs. The manually annotated dataset contains 4,819 labeled skill spans from 327 job postings across different occupation areas. The use of LLM outperforms supervised sequence labeling when used in an end-to-end pipeline, aligning extracted spans with standardized skills in the ESCO taxonomy more effectively. The best-performing configuration, utilizing Claude Sonnet 3.7 with dynamic few-shot prompting for skill identification, embedding-based retrieval, and LLM-based reranking for skill linking, achieves an end-to-end performance of 0.56, positioning Turkish alongside similar studies in other languages, which are few in the literature. Our findings suggest that LLMs can improve skill extraction performance in low-resource settings, and we hope that our work will accelerate similar research on skill extraction for underrepresented languages.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。