对比多个模型在高棉语问答中的表现，发现检索器选择影响更大。
A Comparative Study of Language Models for Khmer Retrieval-Augmented Question Answering
- 用三种嵌入模型检索高棉语文档，BGE-M3表现最佳
- 不同生成模型在事实准确、相关性等指标上各有优势
- 适合低资源语言研究者参考，尤其关注非拉丁文字处理
检索增强生成（RAG）作为一种将大语言模型输出与检索证据结合的范式，能有效减少幻觉并提升事实准确性。然而，该方法在低资源、非拉丁字母语言如高棉语中的效果尚不明确。本文构建了一个基于RAG的高棉语电信领域问答系统。首先，在三个嵌入模型——BGE-M3（567M）、Jina-Embeddings-v3（570M）和Qwen3-Embedding（597M）之间进行密集检索对比，结果表明BGE-M3表现最优，命中率@3为0.285，文件命中率@3为0.700，MRR@3为0.221，精确度@3为0.112。其次，以BGE-M3为检索器，评估五个生成器（Qwen3-8B、Qwen3.5-9B、Sailor2-8B-Chat、SeaLLMs-v3-7B-Chat、Llama-SEA-LION-v2-8B-IT）在200个高棉语问答对的黄金数据集上的表现。采用六项受RAGAS启发的指标：忠实性、答案相关性、上下文相关性、事实正确性、答案相似性和答案正确性。结果显示，无单一模型在所有指标上领先：Qwen3.5-9B在忠实性（0.859）和上下文相关性（0.726）上最高，Qwen3-8B在事实正确性（0.380）上最优，SeaLLMs-v3-7B-Chat在答案相关性（0.867）、答案相似性（0.836）和答案正确性（0.599）上最佳。研究强调，对于高棉语RAG而言，检索器选择仍是主要瓶颈，而生成器性能则取决于任务侧重于真实性、事实精度或语义相似性。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) has emerged as a promising paradigm for grounding large language model (LLM) outputs in retrieved evidence, thereby reducing hallucination and improving factual accuracy. Its efficacy, however, remains largely unexamined for low-resource, non-Latin-script languages such as Khmer. In this paper, we present a RAG-based question answering system for Khmer-language telecom-domain documents. We conduct a two-phase comparative evaluation. First, we benchmark three embedding models: BGE-M3 (567M), Jina-Embeddings-v3 (570M), and Qwen3-Embedding (597M), for dense retrieval over Khmer documents. BGE-M3 consistently performs best, achieving a Hit Rate@3 of 0.285, File Hit Rate@3 of 0.700, MRR@3 of 0.221, and Precision@3 of 0.112, substantially outperforming the other retrievers. Second, using BGE-M3 as the selected retriever, we evaluate five generator backends: Qwen3 (8B), Qwen3.5 (9B), Sailor2-8B-Chat, SeaLLMs-v3-7B-Chat, and Llama-SEA-LION-v2-8B-IT, on a curated golden dataset of 200 Khmer question-answer pairs. To quantify system performance, we apply six RAGAS-inspired metrics: faithfulness, answer relevance, context relevance, factual correctness, answer similarity, and answer correctness. The results show no single model dominates across all metrics: Qwen3.5-9B achieves the highest faithfulness (0.859) and context relevance (0.726), Qwen3-8B attains the highest factual correctness (0.380), and SeaLLMs-v3-7B-Chat performs best on answer relevance (0.867), answer similarity (0.836), and answer correctness (0.599). These findings highlight that retriever choice remains a major bottleneck for Khmer RAG, while generator strengths vary depending on whether the priority is grounding, factual precision, or semantic similarity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。