用印尼社科期刊提升大模型对当地文化的理解
Injecting Knowledge from Social Science Journals to Improve Indonesian Cultural Understanding by LLMs
- 从151本印尼社科期刊提取文本，构建印尼文化语料库IndoSoSci
- 结合RAG与虚构文档查询，在IndoCulture基准上显著超越基线
- 融合维基百科后达新最佳性能，适合文化理解研究者使用
近期对提升大语言模型（LLMs）理解印尼文化的努力日益增多。一个被忽视的重要文化知识来源是本地社会科学期刊，它们可能包含大量来自本土视角的文化研究。本文构建了一个新型文本数据集IndoSoSci，源自151本开放获取的印尼社会科学期刊。我们提出一种有效的方法，将其中的印尼文化知识注入到大模型中：提取与印尼文化相关的事实，并在检索阶段使用大模型生成的假设性文档作为查询，实现检索增强生成（RAG）。该方法在IndoCulture基准测试中显著优于多个强基线模型。此外，通过将IndoSoSci与印尼维基百科结合，我们在该基准上达到了新的最高准确率。
原文摘要 · Abstract (English)
Recently there have been intensifying efforts to improve the understanding of Indonesian cultures by large language models (LLMs). An attractive source of cultural knowledge that has been largely overlooked is local journals of social science, which likely contain substantial cultural studies from a native perspective. We present a novel text dataset of journal article passages, created from 151 open-source Indonesian social science journals, called IndoSoSci. We demonstrate an effective recipe for injecting Indonesian cultural knowledge therein into LLMs: extracting the facts related to Indonesian culture, and apply retrieval-augmented generation (RAG) with LLM-generated hypothetical documents as queries during retrieval. The proposed recipe yields strong performance gains over several strong baselines on the IndoCulture benchmark. Additionally, by combining IndoSoSci with Indonesian Wikipedia, we set a new state-of-the-art accuracy on the IndoCulture benchmark.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。