用压缩长文本的方法提升印地语等低资源语言的问答效果
Long-context Non-factoid Question Answering in Indic Languages
- 通过信息抽取、指代消解等技术缩短上下文，降低计算负担
- 无微调下平均提升4%语义分和47%词级准确率，微调后提升2%
- 适合做多语言低资源场景下的大模型问答系统优化
问答任务在短上下文下对现代大语言模型（LLMs）已较简单，但长上下文因自注意力机制的二次复杂度带来挑战，尤其在印地语等低资源语言中更为显著。本研究探索了开放信息抽取（OIE）、指代消解、答案段落选择（APS）及其组合等上下文压缩技术，以提升问答性能。在四种印地语系语言（印地语、泰米尔语、泰卢固语、乌尔都语）上的实验表明，相较于未压缩的长上下文基线，使用上下文压缩技术在三个主流大模型上实现平均4%的语义得分提升和47%的词级得分提升，无需微调。微调后，语义与词级得分均平均提升2%。此外，该方法显著降低计算开销。解释性分析（如LIME、SHAP）显示，当APS模型自信定位到答案段落后，所选文本内几乎所有词均获高相关性评分。然而研究也指出当前大模型在处理需推理或辩论的非事实型问题时仍存在局限。值得注意的是，将OIE生成的三元组转化为自然语言表达并未提升系统表现。这些发现凸显上下文压缩技术在提升大模型问答效率与效果方面的潜力，尤其适用于低资源语言场景。代码与资源见：https://github.com/ritwikmishra/IndicGenQA。
原文摘要 · Abstract (English)
Question Answering (QA) tasks, which involve extracting answers from a given context, are relatively straightforward for modern Large Language Models (LLMs) when the context is short. However, long contexts pose challenges due to the quadratic complexity of the self-attention mechanism. This challenge is compounded in Indic languages, which are often low-resource. This study explores context-shortening techniques, including Open Information Extraction (OIE), coreference resolution, Answer Paragraph Selection (APS), and their combinations, to improve QA performance. Compared to the baseline of unshortened (long) contexts, our experiments on four Indic languages (Hindi, Tamil, Telugu, and Urdu) demonstrate that context-shortening techniques yield an average improvement of 4\% in semantic scores and 47\% in token-level scores when evaluated on three popular LLMs without fine-tuning. Furthermore, with fine-tuning, we achieve an average increase of 2\% in both semantic and token-level scores. Additionally, context-shortening reduces computational overhead. Explainability techniques like LIME and SHAP reveal that when the APS model confidently identifies the paragraph containing the answer, nearly all tokens within the selected text receive high relevance scores. However, the study also highlights the limitations of LLM-based QA systems in addressing non-factoid questions, particularly those requiring reasoning or debate. Moreover, verbalizing OIE-generated triples does not enhance system performance. These findings emphasize the potential of context-shortening techniques to improve the efficiency and effectiveness of LLM-based QA systems, especially for low-resource languages. The source code and resources are available at https://github.com/ritwikmishra/IndicGenQA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。