arXiv:2607.14349cs.CLcs.AI2026-07ACL

用合成数据训练模型,让逻辑推理摆脱现实常识干扰。

HABIB_TAZ at SemEval-2026 Task 11: Disentangling Formal Logic from Content via Synthetic Training and Multi-Objective Optimization

  • 用规则生成的合成数据微调模型,减少真实语义噪声。
  • 英文与多语言任务中达100%准确率,偏见仅0.00%。
  • 适合研究鲁棒逻辑推理、对抗性测试的学者参考。

大型语言模型在多数自然语言任务中表现优异,但其形式推理能力常受内容影响,表现出对现实合理性的真实偏差。本文针对SemEval-2026任务11,提出系统评估模型在12种语言中分离形式逻辑与内容的能力,包含带干扰前提的情况。我们采用mDeBERTa-v3网络,在基于规则的合成三段论数据集上进行微调,避免大模型增强数据带来的语义噪声。训练流程引入多目标损失函数,结合自适应组分布鲁棒优化(DRO)、可调度的可微分偏置惩罚和KL散度一致性正则化,显式解耦合理性与逻辑结构。系统在子任务1(英语)、子任务2(嘈杂英语)和子任务3(多语言)中均获第一,排名得分100.0,偏见为0.00%,准确率达100.0%。在复杂子任务4(嘈杂多语言)中排名第六,准确率为89.06%,F1为89.06%,偏见仅2.89%,排名得分为37.78。所构建的数据生成引擎与代码库已公开,助力未来鲁棒逻辑推理研究。

原文摘要 · Abstract (English)

While Large Language Models (LLMs) excel in many general NLP tasks, their formal reasoning capabilities are often compromised by content effects, demonstrating a measurable bias towards real-world plausibility. In this paper, we present our system for SemEval-2026 Task 11, which evaluates the ability of models to disentangle formal logic from content across 12 languages with and without distractor premises. We address this challenge using mDeBERTa-v3 networks fine-tuned on a synthetic, rule-based dataset of syllogistic schemes to avoid the semantic noise of LLM-augmented data. To explicitly decouple plausibility from logical structure, our training pipeline employs a multi-objective loss function combining Adaptive Group Distributionally Robust Optimization (DRO), a scheduled differentiable bias penalty, and KL-Divergence consistency regularization. Our system achieved #1 ranks and perfect Ranking Scores (100.0) with 0.00% bias and 100.0% accuracy on Subtask 1 (English), Subtask 2 (Noisy English), and Subtask 3 (Multilingual). On the highly complex Subtask 4 (Noisy Multilingual), the system achieved the 6th rank with 89.06% Accuracy and F1-score, alongside a limited 2.89% Bias and a 37.78 Ranking Score. Our dataset generation engine and codebase are publicly available to facilitate future work on robust logical reasoning.

逻辑推理多语言合成数据偏见控制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。