用大模型生成越南语事实核查数据,提升低资源语言研究能力
Evaluating Large Language Model Capability in Vietnamese Fact-Checking Data Generation
- 通过简单提示词自动构建越南语事实核查数据
- 微调后生成数据质量显著提升但仍不及人工
- 适合低资源语言NLP研究者参考
大型语言模型(LLMs)在阅读理解与推理能力持续提升的背景下,被广泛应用于各类复杂语言任务,包括为多种目的自动生成语言数据。然而,针对越南语等低资源语言的LLM自动数据生成研究仍不充分,缺乏系统评估。本文探索将LLM用于越南语事实核查任务的数据生成,该任务面临严重数据不足问题。具体而言,我们关注从多条证据句合成论断的事实核查数据,以检验LLM的信息整合能力。通过基于简单提示词的自动数据构建流程,并尝试多种方法提升生成数据质量。为评估生成数据质量,我们结合人工评估与语言模型性能测试。实验结果与人工评估表明,尽管经过微调后生成数据质量显著提升,但仍未达到人类生成数据的水平。
原文摘要 · Abstract (English)
Large Language Models (LLMs), with gradually improving reading comprehension and reasoning capabilities, are being applied to a range of complex language tasks, including the automatic generation of language data for various purposes. However, research on applying LLMs for automatic data generation in low-resource languages like Vietnamese is still underdeveloped and lacks comprehensive evaluation. In this paper, we explore the use of LLMs for automatic data generation for the Vietnamese fact-checking task, which faces significant data limitations. Specifically, we focus on fact-checking data where claims are synthesized from multiple evidence sentences to assess the information synthesis capabilities of LLMs. We develop an automatic data construction process using simple prompt techniques on LLMs and explore several methods to improve the quality of the generated data. To evaluate the quality of the data generated by LLMs, we conduct both manual quality assessments and performance evaluations using language models. Experimental results and manual evaluations illustrate that while the quality of the generated data has significantly improved through fine-tuning techniques, LLMs still cannot match the data quality produced by humans.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。