针对阿拉伯语的多跳问答知识编辑系统，提升大模型实时更新能力。
MQA-KEAL: Multi-hop Question Answering under Knowledge Editing for Arabic Language
- 用外部记忆存储结构化知识，分步求解多跳问题
- 在自建阿拉伯语评测集上性能显著优于基线模型
- 填补阿拉伯语知识编辑领域空白，适合多语言研究者
大型语言模型（LLMs）在多个应用领域展现出强大能力，但如何持续更新其知识成为关键挑战。尽管已有诸多关于知识编辑（KE）的研究，且常通过多跳问答（MQA）进行验证，但现有工作主要集中在英文领域。为弥补这一空白，本文提出针对阿拉伯语的多跳问答知识编辑框架MQA-KEAL。该方法将知识编辑以结构化单元存入外部记忆，在回答多跳问题时，先通过任务分解将其拆分为子问题，再迭代查询外部记忆和目标LLM生成答案。同时，我们构建了MQUAKE-AR（英文基准MQUAKE的阿拉伯语翻译版），以及新基准MQA-AEVAL，用于对阿拉伯语环境下知识编辑的多跳问答性能进行严格评估。实验表明，MQA-KEAL在多个指标上显著优于基线模型。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have demonstrated significant capabilities across numerous application domains. A key challenge is to keep these models updated with latest available information, which limits the true potential of these models for the end-applications. Although, there have been numerous attempts for LLMs Knowledge Editing (KE), i.e., to edit the LLMs prior knowledge and in turn test it via Multi-hop Question Answering (MQA), yet so far these studies are primarily focused on English language. To bridge this gap, in this paper we propose: Multi-hop Questioning Answering under Knowledge Editing for Arabic Language (MQA-KEAL). MQA-KEAL stores knowledge edits as structured knowledge units in the external memory. In order to solve multi-hop question, it first uses task-decomposition to decompose the question into smaller sub-problems. Later for each sub-problem, it iteratively queries the external memory and/or target LLM in order to generate the final response. In addition, we also contribute MQUAKE-AR (Arabic translation of English benchmark MQUAKE), as well as a new benchmark MQA-AEVAL for rigorous performance evaluation of MQA under KE for Arabic language. Experimentation evaluation reveals MQA-KEAL outperforms the baseline models by a significant margin.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。