用合成数据测试大模型跨语言推理能力，发现混语反而提升表现
Evaluating Multilingual and Code-Switched Alignment in LLMs via Synthetic Natural Language Inference
- 构建跨语言自然语言推理数据集，控制语义关系并支持混语测试
- 混语场景下模型性能不降反升，翻译带来的词汇差异或起正则化作用
- 适用于关注多语言鲁棒性与跨语言对齐的研究者
大型语言模型在多语言场景中应用日益广泛，但其跨语言间一致且逻辑自洽的对齐能力仍缺乏系统评估。本文提出一种受控的多语言自然语言推理（NLI）评估框架，生成基于逻辑的合成前提-假设对，并将其翻译为语系多样化的多种语言。该设计可精确控制语义关系，支持单语与混合语言（代码切换）两种测试条件。令人意外的是，代码切换并未降低性能，甚至可能提升效果，表明翻译引发的词汇差异或可作为正则化信号。通过嵌入相似性分析与跨语言对齐可视化验证了译文语义保真度。研究揭示了当前大模型跨语言推理的潜力与脆弱性，并指出代码切换是提升多语言鲁棒性的有效途径。代码已开源：https://github.com/KurbanIntelligenceLab/nli-stress-testing
原文摘要 · Abstract (English)
Large language models (LLMs) are increasingly applied in multilingual contexts, yet their capacity for consistent, logically grounded alignment across languages remains underexplored. We present a controlled evaluation framework for multilingual natural language inference (NLI) that generates synthetic, logic-based premise-hypothesis pairs and translates them into a typologically diverse set of languages. This design enables precise control over semantic relations and allows testing in both monolingual and mixed-language (code-switched) conditions. Surprisingly, code-switching does not degrade, and can even improve, performance, suggesting that translation-induced lexical variation may serve as a regularization signal. We validate semantic preservation through embedding-based similarity analyses and cross-lingual alignment visualizations, confirming the fidelity of translated pairs. Our findings expose both the potential and the brittleness of current LLM cross-lingual reasoning, and identify code-switching as a promising lever for improving multilingual robustness. Code available at: https://github.com/KurbanIntelligenceLab/nli-stress-testing
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。