arXiv:2506.04575cs.CL2025-06中稿 · WWW2026被引 3

测试大模型在不同语言表达下的逻辑翻译稳定性,发现一致性差导致推理出错。

Are LLMs Stable Formal Logic Translators in Logical Reasoning Across Linguistically Diversified Texts?

  • 构建多语言形式的逻辑推理基准SoLT,模拟真实文本多样性
  • 引入概念符号映射表方法MenTaL,显著提升符号一致性
  • 实验证明语言差异会大幅降低模型推理准确率,适合关注可靠性研究者

大语言模型在逻辑推理中常将自然语言转为形式逻辑再求解。然而,当同一概念以不同语言形式出现时,模型生成的符号表示常不一致,破坏逻辑连贯性并引发求解错误。现有评测基准缺乏此类语言变异,问题未被充分探索。为此,我们提出SoLT基准,系统地将推理数据集改写为多种语义等价但语言形式多样的版本。此外,我们提出MenTaL方法,在翻译过程中显式构建概念-符号映射表,通过共享符号保持一致性,缓解符号漂移。在SoLT上的实验表明,语言变化导致模型推理准确率显著下降;而使用MenTaL后性能稳定提升。结果揭示:忽视语言多样性会掩盖模型关键缺陷,本工作为实现更可靠的跨语言逻辑推理提供新路径。代码已开源。

原文摘要 · Abstract (English)

Logical reasoning with large language models (LLMs) has received growing attention. One mainstream approach translates natural language into formal logic and then applies symbolic solvers for deduction. While effective in many tasks, these LLM-based translators often fail to generate consistent symbolic representations when the same concept appears in different linguistic forms. Such inconsistencies break logical coherence and lead to solver errors. However, most existing benchmarks lack this type of linguistic variation, which frequently occurs in real-world text, leaving the problem underexplored. To address this gap, we present SoLT, a benchmark that systematically rewrites reasoning datasets into diverse yet logically equivalent forms across multiple levels. Beyond evaluation, SoLT also provides a general method to enrich any dataset with linguistic diversity while preserving both meaning and logic. To further enhance the stability of LLM-based reasoning, we propose MenTaL, which explicitly guides models to build a concept-symbol mapping table during translation. By linking equivalent expressions to shared symbols, MenTaL maintains consistency and mitigates symbol drift. Experiments on SoLT demonstrate that LLMs indeed suffer from inconsistent symbol mapping under linguistic variation, leading to significant drops in reasoning accuracy. Meanwhile, applying MenTaL brings clear and stable performance improvements across diverse inputs. Overall, our findings reveal that overlooking linguistic diversity hides key weaknesses in LLM-based translators, and our work offers a step toward more reliable logical reasoning in varied real-world scenarios. Our code is available at https://github.com/wufeiwuwoshihua/LinguDiver.

逻辑推理语言多样性符号映射大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。