针对非洲低资源语言,研究提示词设计对自然语言推理的影响。
From Script to Semantics: Prompting Strategies for African NLI

- 对比五种提示策略,评估其在三种非洲语言中的表现差异。
- 对抗性提示在不同语言和模型间表现最稳定,提升分类平衡性。
- 精心设计的提示可超越带少样本和思维链的更强模型。
大型语言模型在多语言场景中日益受到关注,但在低资源非洲语言中的推理行为仍缺乏深入研究,尤其在不进行微调的纯提示设置下。本文基于AfriXNLI基准,系统研究了斯瓦希里语、约鲁巴语和豪萨语的自然语言推理(NLI)提示策略,评估了五种策略:基础提示(零样本)、脚本感知、语言特定、对抗性提示及原生标签自翻译(NL-STP),使用两个中等规模开源模型(Llama3.2-3B 和 Gemma3-4B)。为隔离提示设计的影响,研究排除了少样本示例和思维链推理。结果发现,不同策略在类别层面表现差异显著,部分配置存在高度中性类别崩溃与预测偏斜。对抗性提示在跨语言和模型上表现最可靠,显著提升分类平衡性与整体准确率。值得注意的是,精心设计的提示足以超越提供少样本和思维链提示的更强大基线模型。研究证实,提示构造对低资源多语言NLI至关重要,语言感知的决策结构可有效增强资源受限环境下的鲁棒性。
原文摘要 · Abstract (English)
Large language models (LLMs) are increasingly evaluated in multilingual settings, yet their inference behavior in low-resource African languages remains underexplored especially under pure prompting without fine-tuning. We present a systematic study of prompting strategies for Natural Language Inference (NLI) in Swahili, Yoruba, and Hausa using the AfriXNLI benchmark. We evaluate five prompting strategies Baseline (zero-shot), Script-Aware, Language Specific, Contrastive, and Native-Label Self-Translation (NL-STP) across two mid-sized open weight models (Llama3.2-3B and Gemma3-4B). To isolate the effect of prompt design, the effect of few-shot examples and Chain-of-Thought reasoning is eliminated in our study. We find a significant difference in performance of class wise across strategies with highly neutral class collapse and high prediction skew in some configurations. Contrastive prompting proves to be the most reliable and steadily improving strategy over language and model and has better balance of class behavior and balance of overall accuracy gains. Notably, well-constructed prompts are sufficient to beat more powerful baselines that are provided with few-shot prompts and Chain-of-Thought prompts. We have found that prompt formulation is essential to multilingual NLI with low-resource languages and that language aware decision structuring can be used to meaningfully enhance robustness in resource challenged settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。